🔍 Read the full analysis: One Matrix Edit, One Measured Win: GPT-6 Astra Changed A Qwen Model’s Weights And Fixed JSON Tab Escaping From 49 To 70 Of 120 on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
An OpenAI GPT-6 Astra agent edited one matrix in Qwen3-4B-Instruct-2507 and raised its exact-match score on a frozen text-to-JSON test from 49 of 120 cases to 70. The reported gain was limited to that narrow synthetic task: table-row extraction remained at zero of 30, and 50 final-test cases were still wrong.
An OpenAI GPT-6 Astra agent reported that a calculated edit to one matrix in Qwen3-4B-Instruct-2507 raised the model’s exact-match score from 49 to 70 of 120 cases on a frozen test of copying special-character text into JSON. The reported gain—21 additional correct cases—was confined to that synthetic task; it did not improve a separate table-row extraction test.
According to an experiment report published by ThorstenMeyerAI.com on November 5, 2026, GPT-6 Astra, running as a Codex agent, planned and carried out the test. It inspected the model weights, selected the edit, created and froze the test cases, then measured the result. The site says the Qwen model is an open model released under the Apache-2.0 license. The report describes the work as an internal experiment, not a legal or production evaluation.
The test asked the model to place short texts containing tabs and other special characters into JSON without changing their contents. A real tab must be represented in the JSON string as the escape sequence \t and decode back to the original text. The experiment report says the original model answered 49 of 120 final cases exactly; the edited model answered 70. That is a rise of 17.5 percentage points, with 21 cases corrected and no previously correct cases made incorrect, according to the report.
The edit was a calculated rank-one update to the output projection of the model’s final multilayer perceptron layer. The report says no gradient training was used and that the other 397 of 398 stored tensors remained bit-identical to the original. The same report sets limits on the result: the edited model still missed 50 of 120 cases, and table-row extraction scored zero of 30. The test examples were synthetic, and the experiment had not yet received an independent second-AI review.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain in Exact Text Handling
The result matters as a reported example of a model changing another model’s weights and producing a measurable difference on a defined task. Unlike a prompt adjustment, this experiment changed a parameter in the model itself. The reported outcome is specific enough to check: one matrix was edited, and exact answers increased by 21 on a 120-case test.
That does not establish a general improvement in Qwen’s ability to follow instructions or handle structured data. The test covered a narrow failure mode—preserving special characters in JSON—and half of the final cases were still incorrect after the edit. The unchanged table-extraction score also shows that the reported benefit did not carry over to every task the site tested. Readers should treat the result as a single-task experimental finding, not evidence that weight editing reliably improves models more broadly.
The experiment also highlights a practical distinction: producing syntactically valid JSON is not enough when the content must survive a round trip unchanged. If a tab is lost, replaced with spaces, or left unescaped, software receiving the data may get something different from the original text. The test therefore measures a concrete data-integrity requirement, though its synthetic cases do not establish performance on real-world inputs.
Popular right now
De'Longhi Dinamica Plus Fully Automatic Espresso Machine
Samsung 65 The Frame QLED 4K TV (2026) QN65LS03HEFXZA Bundle
Shark HydroVac MessMaster Wet/Dry Vacuum & Mop Combo (AW261)
As an affiliate, we earn on qualifying purchases.
A Third Test in One Week
The reported work follows two earlier experiments described by the same site. In the first, Anthropic’s Claude edited a local model called Ornith; the site reported that more legal answers were completed, while the legal score remained zero. GPT-6 Astra later reviewed that work and identified faults, including a resume bug and command-line problems, according to the site’s account.
Before the Qwen text-to-JSON experiment, GPT-6 Astra had tried a different edit on the same Qwen model, targeting extraction of fields from synthetic business messages. The site says the edited model and original each scored 97 of 200 on the frozen final test. Both produced 47 invalid answers, often by putting raw tab characters into JSON strings. The report says that shared failure prompted the next, narrower experiment; this was a targeted test of a known weakness, not a broad search of model capabilities.
The article says GPT-6 Astra worked as the Codex agent throughout the latest experiment, while Claude had no role in conducting it and was involved in drafting the article. The distinction matters because the reported result comes from the experiment records presented by the site, rather than a published independent replication.
“The starting hint for experiment 3 “came from the previous attempt.””
— The experiment report, as quoted by ThorstenMeyerAI.com
Independent Checks Are Still Pending
The result has not yet been independently reviewed, according to the source. That leaves open whether another evaluator would reproduce the edit, verify the test construction and confirm the reported scores from the stored model and results. The source says its cases are synthetic; it does not establish how the edited model would perform on naturally occurring text or in deployed software.
The published account also does not establish whether the improvement holds across other prompts, model versions, JSON tasks or repeated test sets. Although the final test was described as frozen before the result was seen, the test set’s size is 120 and the task is deliberately narrow. It is therefore not possible from the supplied material to infer a general capability gain, durability of the change, or practical benefit in a production system.
The report says the other 397 stored tensors were unchanged, but the material provided does not include an independent technical audit of the weight files or a replication by another research group. Those checks would help establish how the edit was applied and how broadly the measured effect can be trusted.
Replication Would Test the Result
The next meaningful step is an independent review of the experiment and its records, which the source says has not yet taken place. A reviewer could check the original and edited weights, reproduce the frozen-test scoring, and examine whether the reported 21 corrected cases and zero regressions hold under the stated evaluation procedure.
Further tests would need to establish whether the improvement persists on new examples and whether it transfers to other text-to-JSON inputs without creating failures elsewhere. The source material does not announce a scheduled review, a new experiment date or a planned deployment. For now, the reported finding remains an internal, task-specific result: a one-matrix edit raised the score on one frozen synthetic test, while its wider usefulness remains unverified.
Key Questions
What did GPT-6 Astra change?
The experiment report says the Codex agent applied a calculated rank-one update to the output projection of Qwen3-4B-Instruct-2507’s final multilayer perceptron layer. It reports that the other 397 of 398 stored tensors were bit-identical to the original.
What improved in the test?
On a frozen test of copying text containing tabs and other special characters into JSON, the model’s exact-match result rose from 49 of 120 to 70 of 120, according to the report. The task required the JSON value to decode to exactly the original text.
Did the edit improve the model on other tasks?
The reported table-row extraction result remained zero of 30. The available account does not show a broader improvement across tasks.
Has another group verified the result?
No independent review had been completed at the time of the report, according to ThorstenMeyerAI.com. The test cases were synthetic, and replication on new examples remains an open step.
Source: ThorstenMeyerAI.com
