On November 2, four researchers at MIT CSAIL published a blog post, backed by public logs, showing OpenAI’s GPT-6 Astra reading and rewriting the weights of other neural networks directly, with no optimizer and no gradient descent. In their main test, Astra changed 0.016% of the weights in Qwen3.5-0.8B, a small open model from Alibaba’s Qwen team: 131,072 of roughly 0.8 billion. With a budget of 50 forward passes (test runs of the model on practice problems), Qwen’s score on the MATH-500 benchmark rose from 7.9% to 48.1%.
The post, “Sparks of Weight-Space Programming”, is signed by Rikiya Takehi and Jayden Teoh, its co-first authors, with Alex Zhang and Omar Khattab. The complete logs, including Astra’s edit as a bit-exact diff, are public on GitHub.
Why weights are rarely set by hand
A neural network is a long list of numbers, its weights, almost all set by training: engineers pick the data and a measure of error, and an optimizer such as gradient descent adjusts the weights until the error falls. The authors write that for “almost any human,” handcrafting a network or predicting what one does from its raw weights is “practically impossible.”
Reading weights has so far meant tools built for the job, from interpretability methods to plain statistics: on November 1, Hugging Face users argued that a claimed leak of an Anthropic model was random noise, judging by its weight files. The MIT team asks whether a general-purpose model, used as it is, can work out what a network does from its weights and then change them on purpose, an approach they call Weight-Space Programming.
How Astra found Qwen’s loop
Qwen’s own model card warns that “Qwen3.5-0.8B is more prone to entering thinking loops compared to other Qwen3.5 models, which may prevent it from terminating generation properly.” Asked to make Qwen better at math without training it, Astra got a Python session over the weights with no GPU, about 14,700 practice problems from the GSM8K and MATH training sets, and a budget of forward passes of up to 512 problems each. The test benchmarks stayed hidden from it.
Of the base model’s first 96 practice answers, 85 ran to the 32,768-token limit. Astra traced the looping to one direction written by the MLPs of layers 15 and 19 and subtracted it from 64 output columns in each, touching 131,072 weights in two tensors.
The 50-pass run that took MATH-500 to 48.1% also lifted GSM8K, a grade-school math set, from 13.8% to 67.1%. IFEval, an instruction-following test Astra never aimed at, went from 21.0% to 62.3%; the authors note that the loops are not specific to math. The share of MATH-500 answers stuck at the token limit fell from 93.0% to 33.1%. With 10 or 25 forward passes the loops barely moved, and MATH-500 reached only 12.3% and 14.1%.
The edited checkpoint went up on Hugging Face on November 2 at 22:12 UTC. Its model card reports a re-run, credited in the post to Ahsen Khaliq of Hugging Face, on all 500 MATH-500 problems with four samples each: 2,000 generations in 51 minutes on one Nvidia H200. The edited model scored 51.0%, with 31.6% of answers at the token limit. Only two weight tensors differ from Qwen’s release.
Reading and writing smaller networks
The Qwen edit is the last of eight experiments. In the first, Astra received the raw weights of four game-playing networks as text, 17,253 for each, and named the right game for all four without being given a list. In the third, given only the middle layers of three small language models, it named the language each was trained on and found every copying head, the attention heads that repeat text seen earlier, with a method from Elhage et al. that the authors say they never mentioned to it.
A car-driving network trained with PPO, a widely used reinforcement learning method, had failed in all 256 test episodes to get over a ramp. In one reply, with no tools, Astra edited 3.5% of its 171,664 weights, and the car succeeded in all 1,024 evaluation episodes.
For three games from MinAtar, a suite of simplified Atari games, Astra wrote networks from scratch in a single reply, with no data, no loss function, no gradients and no tools, and none had played before being scored. All three beat PPO networks trained for 100 million steps. In Breakout, Astra’s network scored 517.4, against 25.3 for PPO after 500 million steps, about 20 times as much. In Seaquest it reached 60% of that PPO score, and in Asterix it came close.
The largest network written from scratch plays Go on a 9×9 board. Its 1,480,274 weights all started at zero, and Astra set them in a Python sandbox with no Go library, no internet and no training. With 50 forward passes of up to 128 practice games each, it beat the equal-sized PPO network trained on 20 million games.
A same-day report with Claude
On November 2, Chandan Singh posted a test of the same idea on small tasks with known answers. Claude, as an autonomous coding agent, got a blank transformer and a rule against any training. Across 13 tasks it wrote 959 weight-space programs, and all ten algorithmic tasks reached 100% accuracy within three attempts.
Singh’s write-up is informal, and a note under his byline reads: “Sorry for the AI writing!” His tasks are toys run with a different model, so the report does not confirm the Qwen result.
What the post does not show
The work is a blog post that has not been peer reviewed, and no arXiv paper had appeared by early November 3. The authors call their findings “very early results” and write that the capabilities “are exciting but still limited.” The largest network Astra edited has 0.8 billion weights, a fraction of a frontier model, and the Go network was scored only against PPO networks, with no test against a strong engine or a person.
Two tests failed. Given the weights of two fine-tuned copies of Google’s Gemma 3 270M, one trained to agree with whatever answer the user suggests, Astra could not tell which was the sycophant in four attempts; it spotted it only once it could see attention patterns. Asked to make the same model, which rarely loops, better at math, Astra moved MATH-500 by about a point at most, from 6.8% to 7.7%.
The authors write: “Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked.” The 51.0% re-run, posted on the authors’ own Hugging Face page and done by someone they thank in the post, checks the published checkpoint; the method itself has had no outside review.
Anthropic’s filing the same day
Anthropic filed an amended S-1 with the SEC on November 2. Among its risk factors: “our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI.'” The same passage says it has “in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred.”
The MIT post, which does not mention Anthropic, raises the same theme once, in its outlook, as a possible use: a model able to improve its own weights, “or those of a newer model, offers a richer, more native path” to recursive self-improvement, the authors write. No experiment had a model edit itself; Astra worked on other, much smaller networks.
What it means
Several AI-for-science results this fall were outputs that people could check, such as the proof GPT-6 Astra found for a 24-year-old conjecture or the counterexamples to a 1967 fusion conjecture. The MIT work points the same kind of tool at AI itself, at the part of a model an optimizer almost always writes. Astra read a network it did not train, traced a flaw its maker had documented, and cut the share of math answers stuck at the token limit from 93% to 33%.
What is shown is narrow: weight reading, repairs and from-scratch networks on models up to 0.8 billion weights. The authors hope for more, including audits of weights for hidden backdoors, cheaper optimization and adding or removing skills without new data.
One model editing another directly, without a training run, would go a step past code writing, where AI’s share is already measured: by GitHub’s count, AI agents started more than 40% of the pull requests on its platform in September.
The next markers are the benchmark and harness the authors say they are building, a peer-reviewed paper, replications on larger models by other teams, and whether any lab reports using an LLM to edit the weights of its own production models.
Sources
- Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, MIT CSAIL, November 2, 2026
- RikiyaT/lm_understand-logs, Rikiya Takehi, GitHub, October 29, 2026
- Qwen3.5-0.8B, edited by an LLM with no training (Weight-Space Programming), Hugging Face model card, November 2, 2026
- Qwen3.5-0.8B, Qwen (Alibaba), Hugging Face model card, March 2, 2026
- Weight-Space Programming from scratch on simple tasks, Chandan Singh, November 2, 2026
- Anthropic, PBC, Form S-1/A (Amendment No. 1), Anthropic, PBC, SEC EDGAR, November 2, 2026
Header image: chart by MACCUS, data from Takehi, Teoh, Zhang and Khattab (MIT CSAIL), Sparks of Weight-Space Programming, November 2, 2026
