Of Notes

Bayesian Linear Regression and EM Algorithm

1 min read · revised 22 October 2026 · ·

Table of Contents

Problem

Consider the following Bayesian linear regression model

with known and the prior for is , with known. Now assume that you are given a data set of one training input and the associated response . (a) [25 marks] Using Bayes theorem, derive the expression of the posterior distribution for . (b) [20 marks] Explain what happens in the two limit cases where and . Discuss the interpretation of these two scenarios from the point of view of the trade off between our prior knowledge and what we learn from the data. (c) [20 marks] Describe how the E-M algorithm can be used to estimate both and from the data, via evidence approximation (with known). Define the E-step and the function to be maximised in the M-step. (d) [20 marks] Find an explicit expression for the function as a function of and , where the expectations are over the posterior of . (e) [15 marks] Maximise with respect to , and hence derive the E-M update equation for this parameter.

Solution

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card