Of Notes
Richard Leibler
American mathematician and cryptanalyst of the NSA generation, less famous than his name is. In 1951, with Solomon Kullback, he wrote "On Information and Sufficiency", the paper that defined the divergence the whole field now calls the KL Divergence — a measure of how badly the wrong distribution
- The work note builds it directly from the Shannon Entropy machinery: expand the log, subtract, and the cross-terms collapse.