Artificial Intelligence1 min read

Astra can read and write weights to improve Qwen

By · Published by Everything Blog

In short

The approach, dubbed Weight-Space Programming, allows the AI system GPT-6 Astra to interpret task objectives solely from numerical matrices and author high-performing policies from scratch. It successfully remedied an identified reasoning-loop defect in Qwen3.5-0.8B, drastically enhancing its benchmark scores through targeted numerical edits. Notably, Astra authored small game-playing networks from scratch that surpassed policies trained via standard reinforcement learning for 100 million steps.

Key points

  • Advanced language models can directly inspect continuous weight spaces to enhance interpr…: Advanced language models can directly inspect continuous weight spaces to enhance interpretability without relying on environment feedback.
  • Weight-space programming enables precision parameter editing that fixes architectural fla…: Weight-space programming enables precision parameter editing that fixes architectural flaws without undergoing resource-intensive training loops.
  • Frontier AI systems can program high-performing neural models from scratch faster than co…: Frontier AI systems can program high-performing neural models from scratch faster than conventional reinforcement learning techniques.

Sources checked by our analysis pipeline: 1

We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We…

Artificial Intelligence