Of Notes
World Models in Coding Agents
The idea is something I called CodeJEPA.
- We have a given solution we want to implement
- A coding agent takes multiple turns
where each turn consists of a diff which we can represent indexed over some catalogue of files where is the file in the catalog and it's represented at time and is transformed into with some function is some instruction or direction in which a diff should take the full solution, which is often currently represented in natural language
- The proposal is to create a model that is able to take the current state
and generate candidate solutions in latent space via an auto-encoder like in JEPA models …- Given
, predict ; sample candidates externally; select (see JEPA)
- Given
Figure 1: Diagram