Elements of Statistical Learning

Problem Sheet 3

4 problems · 1 solved

  1. 1.Completing the square for the Gaussian conditional and marginalsolvedsolution

    Use completing the square in the exponent to derive the results in PRML Eqs. (2.141)-(2.142), the conditional and marginal Gaussian formulae.

  2. 2.Eigenvector expansion of a covariance matrix and its inversesolution

    Show that a real symmetric matrix with eigenvector equation (2.45) can be written as an expansion in its eigenvectors with the eigenvalues as coefficients, as in (2.48). Show similarly that has the representation (2.49).

  3. 3.Mean and covariance of a sum of independent random vectorssolution

    Let be independent random vectors, . Show that the mean of is the sum of the means, and that the covariance of is the sum of the covariances.

  4. 3b.Gaussian-gamma conjugate posteriorsolution

    Consider a univariate Gaussian with conjugate Gaussian-gamma prior (2.154) and i.i.d. data . Show that the posterior is again Gaussian-gamma and give its parameters.

Solutions

  1. 1.Completing the square for the Gaussian conditional and marginal

    First problem:

    Likelihood and prior:

    Bayes, dropping the normalising constants:

    Expand both squares:

    Collect by power of :

    so the exponent is , with collecting everything free of :

    Complete the square:

    doesn't depend on , so it goes into the constant; what remains is a Gaussian kernel:

    Match coefficients:

    Precision:

    Mean (multiply top and bottom by , then use ):

  2. 2.Eigenvector expansion of a covariance matrix and its inverse

    No solution yet.

  3. 3.Mean and covariance of a sum of independent random vectors

    No solution yet.

  4. 3b.Gaussian-gamma conjugate posterior

    No solution yet.

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card