Of Notes

Variance and Covariance

1 min read · revised 9 October 2026 · referred to by 4 notes · ·

Table of Contents

A given function has a variance (with respect to a probability distribution over ), which we define as the expected squared deviation of its values from their mean:

If we expand the square, we can show step by step that this is equivalent to another expression. Recall that expectation is linear.

Now we can distribute the expectation using linearity:

Since is a constant, we can pull it out of the expectation in the second term. The expectation of a constant is also just that constant:

Which lets us write the variance in terms of the expectations of and which makes life easier. Written in terms of just :

Covariance

When we have two variables that somehow depend on each other like (for example) the height and weight of a person, we can define the covariance between them as:

Which expresses how much and will vary together. If they are independent, then and if they are positively correlated, then and if they are negatively correlated, then .

In a lot of notations to simplify things, we will write and or for covariance matrices .

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card