Of Notes

Towards a Theory of Dynamical Transferability

1 min read · revised 4 November 2026 · ·

Table of Contents

Can we predict which unfamiliar dynamical systems will provide useful pretraining knowledge for a data-sparse target, before spending resources training on them?

  • Diversification: Explore alternative pretraining domains as an action space, identifying systems with potentially transferable dynamical structures.
  • Reintegration: Transfer knowledge from these domains into a foundation model specialised for a data-sparse target system.
  • Adaptive generalisation: Iteratively search, diversify and reintegrate, using improvements in target performance to guide subsequent exploration.

Specification

A compute-constrained, target-feedback-driven curriculum for repeated diversification and reintegration, designed to discover transferable structure across heterogeneous dynamical domains.

Can iterative, curriculum-guided diversification and reintegration improve data-efficient generalisation in foundation models of dynamical systems compared with conventional sequential pre-training and fine-tuning under equivalent computational budgets?

Questions

  • Can we identify shared dynamical characteristics across different systems that predict successful cross-domain transfer, and how do model architectures and training strategies influence this?
  • Can we learn transferable representations of underlying dynamics from sparse, censored, or otherwise incomplete observations?
    • (Diversification~Reintegration, strong results on Cellular Automata)
  • Can these principles improve the pretraining and generalisation of foundation models, particularly in data-constrained domains such as neuroscience (e.g. fMRI)?
/ or Ctrl K
search; empty, it shows notes to revise and lately read
m
mark this note to revise, or unmark it
c
cover maths, proofs and solutions to test yourself
n
uncover the next covered item
y
copy this note’s org-roam link
e
copy this note as markdown, LaTeX kept as TeX
p
the next problem to work; there h hints, s shows the solution, x passes it for today
→ ← j k space
turn the page, when reading in pages
l
on the map, trace the greatest distance, then the longest path
double-click
a formula, to copy its TeX
rest on maths
bind a part of a formula and open the wheel of notes it comes from; scroll or ↑ ↓ widens it, 1–6 goes
?
this card