Of Notes

Simple Polynomial Curve Fitting

2 min read · revised 20 October 2026 · referred to by 1 note · ·

We start with a simple idea of how to fit a polynomial. We consider an underlying functions which will be the origin of our data.

This data is observations of corresponding with observations of . To each data point we also add random Gaussian noise.

Our goal is to make predictions of which is our target variable based on some new value . For now we fit data:

Although this is a polynomial its called a linear model since the coefficients are linear. This is generally true for any functions with linear unknown parameters.

We obtain our unknowns by minimizing an error function. The most common method is the sum of squares of erros between predictions and data:

We get by minimizing:

First rewrite this as a matrix problem where:

Then we get that which is the same as (which is called the Squared Euclidean Norm). We can expand this:

Now we can take the derivative of this with respect to and set it to zero to find the minimum:

But since is a vector its more proper to use a gradient operator which is the same as .

When we set the gradient to zero we get the Normal Equations.

We can now evaluate the model on new data by computing . We do however have a hyperparameter which is the degree of the polynomial. We will use multiple values and evaluate on a test set to find the best .

We might also use the Root Mean Square Error (RMSE) which is defined as:

/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card