Of Notes
Norm Losses
Table of Contents
If we have some true variable
A general family of norm-based error measures is given by the
The most common cases are
L1 Loss or Absolute Error
For a single prediction,
Over a dataset this gives the Mean Absolute Error (MAE),
Errors are penalized linearly. It is relatively robust to outliers. Minimizing expected L1 loss gives the conditional median,
L2 Loss or Squared Error
In machine learning, ``L2 loss'' usually refers to the squared L2 error,
Over a dataset this gives the Mean Squared Error (MSE),
Large errors are penalized quadratically. It is more sensitive to outliers. Minimizing expected L2 loss gives the conditional mean,
The key relationship is
Note that the literal
whereas MSE uses the squared L2 norm,