I explored a low‑compute technique called Weight‑Space Programming, where a large editor model directly rewrites a tiny fraction of a smaller target model’s weights, yielding outsized performance gains.
I edited only 0.016 % of the target model’s parameters.
The edited Qwen3.5‑0.8B model achieved up to 6× improvement on math benchmarks.
Instruction‑following performance improved up to 3× after the weight edits.
I decided to use a larger model (GPT‑6 Astra) to edit a smaller model rather than fine‑tune the smaller model, to obtain large gains with minimal compute.
Limiting edits to a very small proportion of weights demonstrates that minimal interventions can produce outsized gains.
I noted that this is a less compute operation that will have incremental value in the LLMs world.
I linked this technique to the broader idea of simulating real‑world memory by directly editing model weights. model compression