Of Notes

Discriminative vs Generative Classification and Laplace Approximation

1 min read · revised 17 October 2026 · ·

Table of Contents

Problem

(a) (i) [10 marks] State briefly what is meant by discriminative and generative approaches to classification. List one benefit and one problem with each approach.

(ii) [25 marks] Consider the case of binary classification where we wish to determine if an input point belongs to one of two classes and . If we take a generative approach and find , , using class conditional densities and prior , predictions are made according to Show that can be written in terms of a logistic sigmoid where , for a suitably defined function .

(iii) [25 marks] Assuming the class conditional density is Gaussian, for with expectation and covariance matrix , show that with

(iv) [10 marks] Describe how using basis functions one might extend the above result to more complicated predictors for the class probabilities.

(b) Consider the following supervised learning problem: we are given some data consisting of inputs and output labels . We wish to predict the output label of a new input point , using a discriminative method where is of the form with arbitrary .

(i) [10 marks] Assuming that the labels are conditionally independent given and , where the -th row of is , state the likelihood of the labels given .

(ii) [20 marks] For the case of a zero mean Gaussian prior over with covariance , explain how the Laplace approximation of the posterior is constructed and state the form of the approximate posterior.

Solution

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card