A new demo shows LLM weight editing fixing a stubborn failure in a small model by changing a sliver of its parameters. An AI agent rewrote 0.016% of the weights in Qwen3.5-0.8B and cured its habit of looping forever. There was no training and no gradient descent involved.
The edit removes one “repetition-loop” direction from 131,072 weights out of roughly 800 million. The result is a model that finishes answers it used to abandon.
The problem: a small model that won’t stop checking itself
With thinking turned on, the base Qwen3.5-0.8B keeps second-guessing. It writes lines like “wait, let me verify” over and over until it exhausts its budget instead of giving an answer. On a batch of MATH-500 problems recorded with identical settings, 93.0% of runs hit the 32,000-token limit.
What the agent changed
An LLM agent located a direction inside the model that correlates with these loops. It then subtracted that direction from the output weights of ffn_down in 64 neurons, in layers 15 and 19. Those 64 neurons account for the 131,072 changed weights. Each edited tensor is fed by 3,584 neurons, so the edit touches a small fraction of them.
In the demo’s visual, the edited stripes are drawn about 25 times wider than true scale. At real size they would be invisible.
The numbers after the edit
Same model, same prompts, different weights:
- Runs that hit the token limit fell from 93.0% to 33.1%.
- MATH-500 accuracy rose from 7.9% to 48.1%.
The edited model stops looping and finishes its answers far more often. The accuracy gain is large because a model that never completes a reasoning chain cannot score, so cutting the loops lifts the results directly.
Why the edit seems to work
The LLM weight editing demo looks inside the edited neurons while the model thinks. When the original model starts repeating itself, those 64 neurons fire harder, and the push the edit adds is about 2 to 4 times bigger inside loops than elsewhere in the same run. The edit pushes against what those neurons were writing, so the more they fire, the harder it steers away from the loop.
Running the edited weights over the original model’s looping text gives two more signals. Layer 19 is about half as active, roughly 2.1 compared with about 4.0. And the loop gets shakier: 15.9% of loop tokens fall below 90% probability, against 0.4% for the original model.
The sample behind those inspection charts is small, four loops and one control, and the authors label the finding correlation, not proof. The model was run in bf16 over the recorded tokens and then quantized to int8 for the web.
How it differs from model souping
Model souping, which includes merging diffusion LoRAs, is also a form of weight-space editing. It picks merge weights, sometimes the merge order, using a broad benchmark. LLM weight editing of this kind targets a few specific failure cases and the neurons behind them.
That framing suits agents. An agent can propose an edit, test it, feed the result back and try again, the same edit-test-repeat loop behind recent coding-agent wins.
Try it yourself
Both models run locally in the browser through llama.cpp compiled to WebAssembly and WebGPU, so nothing is sent to a server. The base model is a Q4_K_M quantization built with an importance matrix, and the edited one is Q4_K_M without it. The download is about 1.05 GB (508 MB plus 542 MB) and is cached after the first visit. You can also watch a recording of both models without downloading anything. Sequential runs unload each model by default to limit GPU memory.
What to watch next
This is one small model and one failure pattern, so the open question is whether the method holds up on larger models and on problems other than repetition. The next test is whether the same agent loop can find and fix different failure cases, and whether the edits survive a change in prompts or settings. For now, the demo is a clear, reproducible example that behavior-level fixes can live in a few dozen neurons.
Source: Weight-Space Playground