There’s something fascinating happening in the world of AI at this moment, and hardly anyone talks about it. AI can improve itself without training. Say that again? Yes! It turns out frontier models are capable of changing other, smaller models, by just editing their weights. Meaning they just look at the final file containing the model, that collection of matrices, they “understand” what’s there and they can take action, moving parts around, deleting them, editing them, etc.

That’s huge. It really is. I recorded a small video with my short term and long term implications of this, if you’re the visual type (or keep reading if you’re not):

It all came from an experiment at MIT CSAIL, in which researchers took a large model (Astra) and pointed it at a much smaller one — Qwen 0.8B. They didn’t fine-tune anything. They just asked the big model to look inside the small model’s weights and see if anything could be improved.

Astra found a section of weights that was effectively blocking better math performance and removed it. After that single edit, the 0.8B model got substantially better at math. No training, no additional data, just a surgical change based on what the larger model could see in the weights.

They also ran a recognition test. Small models were trained on simple games (Pac-Man and others). The big model was given only the weights and asked to figure out what the small model was doing. It identified the correct game with over 70% probability. Again, no examples, no execution — just inspection of the weights.

Someone later tried a similar recognition task with the open-source Qwen 3.8 27B. It reached the same conclusion, though it took roughly 40 minutes instead of the faster result from the larger proprietary model.

What Self-Improving AI Can Realistically Do?

The researchers observed that this ability to usefully read and edit weights appeared “abruptly”. Nobody had a clear explanation for why it showed up when it did.

In the near term this is already interesting for practical work. If you have a fine-tuned small model that isn’t quite right, you can now hand it to a stronger model and ask it to look for problems or improvements without starting another training run. You can also get a first-pass audit of a small model’s likely behavior by having a larger one inspect the weights rather than running extensive tests.

Going forward, the open question is whether frontier models can eventually generate useful smaller models more directly — essentially producing the weights instead of only editing existing ones. If that becomes reliable, it removes a large chunk of the usual training compute from the process of making practical models.

That would affect how much large-scale GPU infrastructure is actually required for many use cases, and it would change the economics for the biggest labs. These are still speculations right now; the demonstrated result so far is inspection and targeted editing, not full model generation from scratch.

But the “abrupt” appearance of the capability hints at something bigger. Similar jumps have happened before with other abilities. If this one continues to develop, it could shift how we think about the cost and process of getting capable models into practical use.

I’ll keep an eye on the development of these experiments, and, if time and resources allow, I will try to reproduce.

If you like this kind of videos, like, share and subscribe to my YouTube channel!