Consider Gaussian Processes (GPs) on inputs .
(a) [5 marks] State the definition for a stochastic process to be a GP with mean function and covariance function . Explain what it means for the covariance function to be valid.
(b) [5 marks] Show that the covariance function is valid.
(c) [5 marks] Show that, given two valid covariance functions and , is valid.
(d) [10 marks] Now consider a Gaussian process regression , where for a one-dimensional input with the kernel function
and you are given one input value and corresponding observation . Compute the predictive distribution in and discuss what happens to the uncertainty of the prediction in the limit .