Of Notes

Mixture Distributions, EM Algorithm, and K-Means Connection

1 min read · revised 17 October 2026 · ·

Table of Contents

Problem

Consider a random variable with a mixture distribution of the form where the conditional expectation of is and the conditional covariance matrix of is . Let be an i.i.d sample from this mixture model.

(a) [20 marks] Derive the expressions of the expectation and covariance matrix of .

(b) [25 marks] Outline how the EM algorithm can be used to estimate the parameters .

Let us now assume that have multivariate normal distributions with mean and covariance matrix , with the identity matrix.

(c) [35 marks] Derive the expressions of the update formulas for , and in the M-step of the algorithm.

(d) [20 marks] Assume now that is known. Comment on what happens to the EM procedure when and what similarities there are between the resulting algorithm and the K-means clustering.

Solution

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card