Consider the following Bayesian linear regression model
with known and the prior for is , with known.
Now assume that you are given a data set of one training input and the associated response .
(a) [25 marks] Using Bayes theorem, derive the expression of the posterior distribution for .
(b) [20 marks] Explain what happens in the two limit cases where and . Discuss the interpretation of these two scenarios from the point of view of the trade off between our prior knowledge and what we learn from the data.
(c) [20 marks] Describe how the E-M algorithm can be used to estimate both and from the data, via evidence approximation (with known). Define the E-step and the function to be maximised in the M-step.
(d) [20 marks] Find an explicit expression for the function as a function of and , where the expectations are over the posterior of .
(e) [15 marks] Maximise with respect to , and hence derive the E-M update equation for this parameter.