Of Notes

Discriminative Classification and Laplace Approximation

1 min read · revised 22 October 2026 · ·

Table of Contents

Problem

Consider the following supervised learning problem: we are given some data consisting of inputs and output labels . We wish to predict the output label of a new input point , using a discriminative model where is of the form with parameters and is a known function. (a) [20 marks] Assuming a loss function given by the matrix , where (= loss when true class is but we predict ) and (= loss when true class is but we predict ) and are known, derive the decision rule to classify a new input . (b) [10 marks] Assuming that the labels are conditionally independent given and , where the $i$-th row of is , state the likelihood of the labels given . Let now the function be the logistic sigmoid

(c) [20 marks] Show that the gradient of the log-likelihood has the form , and find out the expression of . (d) [20 marks] Show that the Hessian matrix of the log-likelihood has the form , for the appropriate choice of the matrix . Show then that has a unique maximum (i.e. that its Hessian is negative definite). (e) Let's now assume the parameters to have a prior distribution which is a multivariate normal with zero mean and covariance matrix . (i) [15 marks] Show that the posterior distribution for is not normal but it is unimodal. (ii) [15 marks] Describe how to derive the Laplace approximation for this posterior distribution.

Solution

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card