Mathematical Foundations for Complex Systems Modelling
Large Deviations
1.Chebyshev against the normal approximation for a Poisson sample meansolution
Let
be independent Poisson random variables with parameter . Call .a) Using the Chebyshev inequality, give an estimate for
. b) Now estimate the probability above using the normal approximation (you can express the result in terms of the function ). c) Compare the estimates when , and .2.Is a coin that shows 64 heads in 100 tosses fairsolution
A coin is tossed 100 times and the number of times it lands on head,
, equals , and the number of times it lands on tail, , is . Is the coin fair? To answer this question use the normal approximation to estimate assuming that the coin is fair.3.Joint pmf and rate function for three-party voting countssolution
The MaxEnt probabilities for the voting patterns
in the independent three-party voters problem discussed in the previous problem, depend only on the total numbers , and of , and votes and can be written as where and are the single-voter marginal probabilities to vote , and , given by(a) Verify the expressions for the marginal probabilities
and .(b) Using the expression for
and the fact that it depends only on , derive the joint pmf of the numbers of votes cast for the three options. [Hint: you will need the number of ways to partition a set of elements into 3 non-overlapping subsets of sizes , i.e. a multinomial coefficient.](c) Use the Stirling approximation of the factorials in the multinomial coefficient of (b) to show that the joint pmf
for the fractions , is for approximated by in which the rate function is(d) From the structure of the expression for
in terms of the rate function, argue that in the limit the pmf is exponentially small in unless . [Hint: the reasoning is completely analogous to that used in the two-party voter case.]4.Random points in the unit square and the law of large numberssolution
is an infinite sequence of random points chosen independently and uniformly inside the square . For every , define the random variable equal to if and equal to if . Find5.Multinomial coefficients and entropysolution
The number of ways of arranging
balls in a line, where the balls have different colours and there are balls of colour , is the multinomial coefficient . Use the Stirling approximation to show that, to exponential accuracy, this is , where and is the entropy in natural units of the distribution .6.Boltzmann's argument for maximum entropy inferencesolution
Consider a three-party voter whose voting decision we call
. Over a large number of elections, it has been found that . Let be the number of times the voter has voted for each of the three parties in these elections; we want to estimate , and similarly and .The number of voting histories that give vote counts
is . Use Stirling's formula to evaluate the large- behaviour of this multinomial coefficient. Note that, out of all possible values of consistent with , the ones for which there are overwhelmingly the largest number of voting histories, and which should therefore occur almost certainly, are those where maximise . This argument for maximum entropy inference is due to Boltzmann. Explore this result numerically for the specific case ; assume that is a multiple of .
Solutions
1.Chebyshev against the normal approximation for a Poisson sample mean
No solution yet.
2.Is a coin that shows 64 heads in 100 tosses fair
No solution yet.
3.Joint pmf and rate function for three-party voting counts
No solution yet.
4.Random points in the unit square and the law of large numbers
No solution yet.
5.Multinomial coefficients and entropy
No solution yet.
6.Boltzmann's argument for maximum entropy inference
No solution yet.