Elements of Statistical Learning
Problem Sheet 3
1.Completing the square for the Gaussian conditional and marginalsolvedsolution
Use completing the square in the exponent to derive the results in PRML Eqs. (2.141)-(2.142), the conditional and marginal Gaussian formulae.
2.Eigenvector expansion of a covariance matrix and its inversesolution
Show that a real symmetric matrix
with eigenvector equation (2.45) can be written as an expansion in its eigenvectors with the eigenvalues as coefficients, as in (2.48). Show similarly that has the representation (2.49).3.Mean and covariance of a sum of independent random vectorssolution
Let
be independent random vectors, . Show that the mean of is the sum of the means, and that the covariance of is the sum of the covariances.3b.Gaussian-gamma conjugate posteriorsolution
Consider a univariate Gaussian
with conjugate Gaussian-gamma prior (2.154) and i.i.d. data . Show that the posterior is again Gaussian-gamma and give its parameters.
Solutions
1.Completing the square for the Gaussian conditional and marginal
First problem:
Likelihood and prior:
Bayes, dropping the normalising constants:
Expand both squares:
Collect by power of
:so the exponent is
, with collecting everything free of :Complete the square:
doesn't depend on , so it goes into the constant; what remains is a Gaussian kernel:Match coefficients:
Precision:
Mean (multiply top and bottom by
, then use ):2.Eigenvector expansion of a covariance matrix and its inverse
No solution yet.
3.Mean and covariance of a sum of independent random vectors
No solution yet.
3b.Gaussian-gamma conjugate posterior
No solution yet.