Of Notes

Norm Losses

1 min read · revised 22 August 2026 · referred to by 1 note · ·

If we have some true variable and an estimate of that variable , we define the error as

A general family of norm-based error measures is given by the norm

The most common cases are , giving the and norms.

L1 Loss or Absolute Error

For a single prediction,

Over a dataset this gives the Mean Absolute Error (MAE),

Errors are penalized linearly. It is relatively robust to outliers. Minimizing expected L1 loss gives the conditional median,

L2 Loss or Squared Error

In machine learning, ``L2 loss'' usually refers to the squared L2 error,

Over a dataset this gives the Mean Squared Error (MSE),

Large errors are penalized quadratically. It is more sensitive to outliers. Minimizing expected L2 loss gives the conditional mean,

The key relationship is

Note that the literal norm is

whereas MSE uses the squared L2 norm, , up to the factor .

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card