Elements of Statistical Learning

Summer 2023 Examination

4 problems · 0 solved

  1. 1.Univariate Normal MLE, MAP, and Empirical Bayessolution

    Consider an independent and identically distributed sample from a univariate Normal distribution

    (a) [5 marks] Derive the expression of the maximum likelihood estimator for the mean . (b) [7 marks] Assuming known and a prior distribution , derive the expression of the posterior distribution for . (c) [13 marks] Explain how an Empirical Bayes approach can be used to estimate the variance parameter (now assumed to be unknown) and and derive the expression of the marginal likelihood to be maximised.

  2. 2.K-Means Clustering, Silhouette Index, and GMMsolution

    You measured the following data points:

    (a) [5 marks] Using the Euclidean distance in and the starting representatives , you are asked to consider the K-means clustering with . How many iterations will be required for the 2-means algorithm to converge and what is the final clustering assignment? (b) [5 marks] Show that the 2-means algorithm would return a different clustering assignment for other choices of the starting representatives. (c) [8 marks] Using two clustering assignments obtained at point (a) and (b) from different starting representatives, compute the silhouette index for the point . Comment on how well the two assignments are clustering this point. (d) [7 marks] Define a Gaussian mixture model to cluster the inputs above in clusters. Describe one advantage of this approach over the K-means (for a generic sample, not necessarily the one considered in this question).

  3. 3.Generative Classification and Decision Rulesolution

    You are tasked to develop a classifier for a set of univariate observations with 3 possible classes, using the generative model

    and the loss function given by the matrix

    (a) [7 marks] Assuming the parameters known, derive the expression of the decision rule to classify a new observation . (b) [10 marks] Derive the expression of the marginal distribution . (c) [8 marks] Now let . How would you classify the new observation ? Should you trust this prediction? Why?

  4. 4.Gaussian Processes and Kernel Validitysolution

    Consider Gaussian Processes (GPs) on inputs . (a) [5 marks] State the definition for a stochastic process to be a GP with mean function and covariance function . Explain what it means for the covariance function to be valid. (b) [5 marks] Show that the covariance function is valid. (c) [5 marks] Show that, given two valid covariance functions and , is valid. (d) [10 marks] Now consider a Gaussian process regression , where for a one-dimensional input with the kernel function

    and you are given one input value and corresponding observation . Compute the predictive distribution in and discuss what happens to the uncertainty of the prediction in the limit .

Solutions

  1. 1.Univariate Normal MLE, MAP, and Empirical Bayes

    No solution yet.

  2. 2.K-Means Clustering, Silhouette Index, and GMM

    No solution yet.

  3. 3.Generative Classification and Decision Rule

    No solution yet.

  4. 4.Gaussian Processes and Kernel Validity

    No solution yet.

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card