Artificial Intelligence1 min read

The 'Astra can read and write weights to improve Qwen' post is misleading

By · Published by Everything Blog

In short

Recent reports highlighted that Astra raised Qwen3.5-0.8B's score on MATH-500 from 7.9% to 48.1% by spotting reasoning loops without gradient descent. However, this benchmark relied heavily on external Python tooling and up to 50 practice problem passes rather than pure parameter inspection. Furthermore, direct evaluation of raw weights in context was confined exclusively to tiny networks of up to 172,000 parameters, and a replication attempt on Gemma 3 failed.

Key points

  • The claims inaccurately combined tool-free weight reading tests on tiny models with compl…: The claims inaccurately combined tool-free weight reading tests on tiny models with complex language model engineering.
  • Full language model evaluations consistently required execution tools rather than relying…: Full language model evaluations consistently required execution tools rather than relying entirely on context-window weight inspection.
  • Astra behaves like a human machine learning engineer by writing analytical code rather th…: Astra behaves like a human machine learning engineer by writing analytical code rather than acting as an autonomous weight editor.

Sources checked by our analysis pipeline: 1

Stage magicians often do the same trick multiple ways. One iteration uses a gimmick prop, the next one shows you the prop isn’t gimmick but that one palms the object, the third time was a forced choice etc. When you watch the performance you assume there’s only one technique, so each time they do it they seem to debunk your hypothesis about how it must be working.

This is pretty much how the experiments in the ‘Astra can read and write weights to improve Qwen’ post work. The post’s abstract says:

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises…

Artificial Intelligence