Consider Gaussian Processes (GPs) on inputs .
(a) [10 marks] State the definition for a stochastic process to be a GP with mean function and covariance function . Explain what it means for the covariance function to be valid.
(b) [10 marks] Show that the covariance function is valid, if is a symmetric positive definite matrix.
(c) [10 marks] Show that, for an arbitrary function , the covariance function is valid.
(d) [15 marks] Consider two independent zero-mean GPs and with covariance functions and respectively. Let , and let be the corresponding Gram matrix. Show that is a valid covariance function.
(e) Now consider a Gaussian process regression , where is a Gaussian process.
(i) [15 marks] Find the marginal distribution of .
(ii) [20 marks] Consider predicting a new observation at . By considering the joint distribution of and or otherwise, find the predictive distribution of .
(f) Let for a one-dimensional input with the kernel function
and you are given one input value and corresponding observation .
(i) [10 marks] Compute the predictive distribution in and discuss what happens in the limit .
(ii) [10 marks] Compute the predictive distribution in and discuss what happens in the limit .