Of Notes

Gaussian Integrals and the Laplace Method

15 min read · revised 20 October 2026 · referred to by 2 notes · ·

Week 3 of Mathematical Foundations for Complex Systems Modelling. The goal is to compute integrals of the exponential of a quadratic, and then to use them to approximate much harder integrals. Everything below is built from zero, and every change of variables is carried out step by step.

The Basic Gaussian Integral

We want to compute:

where is a constant which makes the exponent negative, so the integrand dies off at and the integral exists.

There is no antiderivative of in elementary functions, so we compute indirectly.

The key observation is that is a single number, not a function of anything. So we are allowed to multiply it by itself, and that is all means:

Now we have two integrals which both use the letter . That is a collision, so we rename the integration variable in the second integral to . This changes nothing about its value, because the integration variable is a dummy: the value of does not depend on what we call the letter we integrate over, any more than depends on the letter :

Next, the product of two integrals over independent variables is the double integral of the product of the integrands. To see why, fix : while we integrate over , the whole first integral is just a constant multiplying , and a constant can be pulled inside the integral. Doing this in both orders:

Applying this with and , and combining the two exponents into one using :

So squaring did two things at once: it gave us permission to introduce a second variable , and the exponent now depends on the pair only through the distance from the origin — which is exactly what will let us use polar coordinates next.

Change of Variables to Polar Coordinates

The exponent only depends on the radius , so we switch to polar coordinates.

The change of variables and its inverse are:

Substituting into the exponent directly gives , so the exponent becomes .

The area element transforms as , where the extra factor is the Jacobian determinant of the map , i.e. the determinant of the matrix of partial derivatives:

The whole plane corresponds to running from to and running from to , so:

The integral gives because the integrand does not depend on the angle:

Change of Variables in the Radial Integral

We substitute .

Differentiating both sides with respect to gives , which we write as the differential relation:

so the measure appears on the right-hand side, ready to be replaced.

The bounds transform too: when we get , and as we get , so the bounds stay to .

Replacing and :

where the remaining integral is one because .

Since we take the positive root:

Dividing both sides by gives the normalized form used in the lecture notes (Eq. 4.26):

which says the function has total area one, i.e. it is a probability density.

Adding a Linear Term

Next we add a linear term to the exponent:

We complete the square. We expand with the binomial formula and compare it to what we have:

So our exponent equals the square minus too much by , and we add it back:

where the last term is a constant, so it factors out of the integral.

Change of Variables: the Shift

We shift , so .

Differentiating gives , so the measure is unchanged:

The bounds map to , which are still : shifting by a finite constant does not move an infinite bound.

The square term becomes , and the constant factors out:

where the last equality is the basic Gaussian integral from the first section.

In the normalized form of the lecture notes (Eq. 4.25), this is called the Gaussian linearization identity (or Gauss–Hubbard–Stratonovich transformation):

which linearizes quadratic exponents in partition function calculations: the quadratic in the exponent trades against the linear term to produce a pure exponential of .

Several Dimensions

Now let be a vector in and an real symmetric positive-definite matrix (all eigenvalues , see Eigen Values & Vectors), so that the quadratic form is positive for every .

We compute:

Change of Variables: the Matrix Square Root

The matrix can be diagonalized by an orthogonal matrix , i.e. where holds the eigenvalues of (Spectral Decomposition).

Since every , we can define the matrix square root and its inverse:

where simply takes in each diagonal slot.

Two facts about these matrices, which follow from multiplying out the diagonal form:

The second one holds because diagonal matrices commute: .

We change variables to , so .

For a linear change of variables the Jacobian matrix has entries , i.e. the Jacobian matrix is , so the volume element transforms with its determinant. This is the $n$-dimensional generalization of the factor we computed for polar coordinates:

Here , and we compute its determinant from the product rule . For the orthogonal matrices, , and for a diagonal matrix the determinant is the product of the diagonal entries:

where is the product of eigenvalues.

Now we transform the exponent. Substituting and using the commuting identity above:

So the coupled quadratic form in becomes a sum of independent squares in , and the integral factorizes into copies of the basic Gaussian integral with :

Putting the Jacobian back:

With a Linear Term

We add a linear term .

Since is invertible, we complete the square with matrices. By analogy with dividing by in one dimension, the shift to complete the square with is the solution of , i.e. .

We multiply out using the bilinearity of the quadratic form, which gives four terms:

and we evaluate the three correction terms one at a time.

The first collapses because :

The second is the transpose of the first. A transpose of a scalar is the same scalar, so as well.

For the last one we use the symmetry of , which implies :

Putting the four terms back together and multiplying by :

So our exponent equals the square minus too much by , and we add it back. The identity we need is:

which is the matrix version of .

Change of Variables: the Shift

We shift , so .

Differentiating, , so the Jacobian matrix is the identity and:

The domain is all of shifted by a constant vector, which is still all of .

The square term becomes , and the constant factors out:

where the last equality is the multivariate Gaussian integral from the previous section.

This is the fundamental multivariate Gaussian identity.

We can now derive the density of the multivariate Gaussian from it. We look for a probability density of the form , centred on the mean vector , with a constant still to be determined.

We fix by demanding that the total probability is one, . We shift variables , so . The shift is by a constant vector, so , and the domain stays all of :

where we used the multivariate Gaussian integral from the previous section. Solving for :

Setting (the inverse covariance, or precision matrix), this gives the Random Variable density of the multivariate Gaussian:

The Laplace Method

We now invert the logic: instead of computing Gaussian integrals exactly, we use them to approximate integrals which are not Gaussian.

We consider:

where is a large parameter, has a unique global minimum at an interior point with and , and .

Why the Minimum Dominates

The integrand is largest where is smallest, because the exponent is least negative there.

At a point where , the integrand is smaller by a factor , which collapses to zero for large .

So for only the neighbourhood of contributes, and everything depends on how behaves near its minimum.

Taylor Expansion Around the Minimum

Since only matters, we expand both functions in a Taylor Expansion around .

The Taylor expansion of a function around a point replaces it by a polynomial in the distance , whose coefficients are the derivatives of the function at :

We apply this to and to with .

For the linear term vanishes by the definition of a stationary point, , so the leading correction is quadratic:

For we keep only the constant term. The next Taylor term would be , which is odd around while the Gaussian it multiplies is even, so its integral cancels; the surviving corrections only produce the at the end and never grow with :

Substituting both into the integral:

where the constant factors out because it does not depend on .

Change of Variables: the Rescaling

The remaining integral is almost the basic Gaussian integral, except for the width in the exponent.

We define by:

Differentiating with respect to gives , so:

The bounds transform into:

Since is interior, and are fixed negative and positive numbers, so as we get and , and we may extend the integral to the whole real line.

The exponent transforms as:

where the cancels exactly between the width and the rescaled variable.

Putting the transformed measure, bounds and exponent together:

where the last equality is the basic Gaussian integral with .

The 1D Laplace Formula

Collecting everything (lecture notes Eq. 4.17):

where denotes corrections which vanish as , coming from the higher terms of the Taylor expansions which we dropped.

Several Dimensions

The same derivation works over a domain , with . The role of is now played by the Hessian matrix , the matrix of second derivatives, which must be positive-definite at the minimum.

The scalar Taylor expansion stacks into vector form, and again the linear term vanishes because is stationary, so the leading correction is the quadratic form:

Change of Variables: the Rescaling

Exactly as in one dimension we define the rescaling variable by:

The change of variables is linear with matrix , so the volume element transforms with . We use for a scalar multiplying an matrix :

The exponent cancels exactly as in one dimension. Substituting into the quadratic Taylor term, and using from the spectral form of the square root:

Since is an interior minimum, the rescaled domain grows into all of as . Collecting the pieces, and evaluating the integral with the multivariate Gaussian integral:

The result collects as:

where replaces , exactly as replaced in the multivariate Gaussian.

Application: Stirling's Approximation

As a check that the machine works, we derive the asymptotics of the factorial.

We start from Euler's integral representation:

To check that this integral really is , we integrate by parts with and , so and :

The boundary term vanishes at both ends: beats any power of as , and at . So the integral for equals times the integral for , and since , repeating this step times produces exactly .

The integrand does not have the shape yet: the large parameter sits inside the power , not in front of the exponent.

Change of Variables: Pulling N Out

We substitute , so .

Differentiating, , so:

The bounds stay and since scaling by a positive constant does not move them.

Now each factor transforms separately. The power becomes , and the exponential becomes . Substituting everything:

where the from the power and the from the measure combine into .

To move into the exponent we write :

Now it has the shape with and .

Finding the Minimum

We find the stationary point:

and evaluate the ingredients. The value at the minimum:

The second derivative:

Plugging into the Formula

The 1D Laplace formula gives:

Multiplying back by the prefactor :

where we combined and .

which is James Stirling's approximation, and the factorial is exactly the kind of integral where no elementary antiderivative exists, so this is the only way to get at its large- behaviour.

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card