Mathematical Foundations for Complex Systems Modelling
Week 4
1.Moment generating functions of the Poisson and exponential lawssolution
(i) A Poisson random variable
with parameter is a discrete RV taking non-negative integer values with probabilities Find the moment-generating function and use it to deduce the mean and the variance .(ii) A continuous RV
has exponential distribution with parameter if its density is for and for . Find the moment-generating function and use it to deduce the mean and the variance .2.The Dirac delta as a Gaussian limit and its Fourier representationsolution
(i) Verify that
(ii) Combine the regularisation (1) of the Dirac delta-function with the Gaussian integral identity (see Week 3 lectures)
to verify the Fourier integral representation(iii) Verify that
for all , .3.Characteristic function of the simple Bernoulli random walksolution
A simple Bernoulli random walk of a particle on
is defined by two rules: the initial position is deterministic, ; and in equal intervals of time the particle goes either one step right with probability or one step left with probability .Denoting by
the position after steps, show that(i) the characteristic function
satisfies(ii) Find
and use it to deduce that the probability density for the position after steps is where is the binomial coefficient.4.The Wiener process density solves the diffusion equationsolution
The Wiener process
(aka Brownian motion) is characterised by: and is continuous in with probability 1. has independent increments: for all the future increments , , are independent of the past values .- The increments are Gaussian:
is normally distributed with mean and variance , .
Use the characteristic function
of the value at time to deduce that its probability density satisfies the diffusion equation Hint: exploit the Fourier relations and .5.A bivariate Gaussian density: normalisation, marginals and covariancesolution
Given a
positive definite real symmetric matrix , consider the 2-component random vector whose joint probability density function (j.p.d.f.) is(i) Verify that the j.p.d.f. is correctly normalised:
.(ii) Show that both
and are Gaussian with mean zero and variances(iii) Further show that the covariance is
and hence deduce that vanishing covariance indeed implies independence in the Gaussian case.(iv) Assume that
and are characterised by the Gaussian j.p.d.f. (4). Prove that6.The Dirac delta of a function with isolated zerossolution
Sketch a proof of the identity
where are all isolated zeroes of , such that and for all . Hint: exploit the identity for all .7.Sums of independent variables and the central limit theoremsolution
(i) Assume that
and are two independent random variables with p.d.f.'s and . Use the characteristic function to show that the p.d.f. of is and give an interpretation of the result. Hint: is the inverse Fourier transform of the characteristic function, ; then use the convolution formula for Fourier transforms.(ii) Prove the Central Limit Theorem: for a set
of independent random variables with the same distribution, with p.d.f. , mean and finite variance , the sample mean is distributed according to the normal law as . Hint: consider the characteristic function and its factorisation for independent variables, and apply the Laplace method to find its asymptotics as . Then compare with the characteristic function of the normal distribution.8.Symmetric random walk: exact law and Gaussian limitsolution
Consider the symmetric Bernoulli random walk defined in Problem 3 (
).(i) Use expression (3) to show that the probability for
to equal a given integer is and zero otherwise.(ii) By considering the limiting shape of (5) for
keeping , and recalling the Stirling formula for (see Week 3 of the lecture notes), show that for . Discuss the meaning of this result.9.Jensen's inequalitysolution
The function
is strictly convex if for all the graph of lies entirely above its tangent at . Mathematically: The function is strictly convex iff its second derivative is non-negative. If in (6) the strict inequality is replaced by , the function is called convex (the graph may then have "flat" parts).(i) Let
be a real random variable. Verify that for any convex function the Jensen inequality holds: Hint: let and be random variables such that (equivalently, is a non-negative random variable). As the expectation of a non-negative random variable is non-negative, it follows that : taking expectation preserves inequalities.(ii) Let
be a discrete random variable taking values with probabilities . Show that for any convex function(iii) Let
be any function such that is well defined for any . Show that for any real random variable with the inequality being strict for a strictly convex function , unless is a constant.10.Lorentzian delta sequence and the Dirac combsolution
Verify that
(i)
, where .(ii) Use the above to show that for any real
Hint: replace in the left-hand side for , resum the resulting geometric series and consider the limit , exploiting the identity , where are such that , .11.Bernoulli variable moments and the random-walk covariancesolution
(i) A Bernoulli random variable
is a discrete RV taking values with probabilities , . Verify the following properties:(ii) Denoting by
the position of a particle after steps in the simple Bernoulli random walk, show that
Solutions
1.Moment generating functions of the Poisson and exponential laws
No solution yet.
2.The Dirac delta as a Gaussian limit and its Fourier representation
No solution yet.
3.Characteristic function of the simple Bernoulli random walk
No solution yet.
4.The Wiener process density solves the diffusion equation
No solution yet.
5.A bivariate Gaussian density: normalisation, marginals and covariance
No solution yet.
6.The Dirac delta of a function with isolated zeros
No solution yet.
7.Sums of independent variables and the central limit theorem
No solution yet.
8.Symmetric random walk: exact law and Gaussian limit
No solution yet.
9.Jensen's inequality
No solution yet.
10.Lorentzian delta sequence and the Dirac comb
No solution yet.
11.Bernoulli variable moments and the random-walk covariance
No solution yet.