Of Notes
Multivariate Gaussian Distribution
Table of Contents
Previsouly when we had
We can now write as:
I prefer the vector notation of vector values in this distribution because it's easier to see.
Intuitively lets start with
By the product of exponentials:
The key step comes in transforming this into a vectorized form:
First we represent the differences as a vector of differences:
Turning it into a row vector:
Now to introduce the variance we define:
Of which the inverse is:
Now we have nicely isolated the numerator and denominator terms of the summation. We can reconstruct this fully by multiplying:
You will notie a missing square in the numertor, we fix:
This is called the Quadratic Form in Matrices.
We now have a very early form of the multivariate gaussian. What we recovered in this last term is what you would call the Mahalnobis distance:
Properties of Variance
The name for the inverse of the variance is yet again what we call precision:
We also note that the number of free parameters in a full variance matrix grows as
- Full General
- Isotropic Variance
- Diagonal Covariance
Eigenvalue Decomposition of Covariance Matrix
We start with the intuition of how eigenvectors and eigenvalues describe the shape of a Gaussian:
Looking at the exponential term we perform a projection of
Sufficient Statistics
Generally for the Gaussian: