Models Lose the Plot, Laptops Read Weights · Morning Gist
Wednesday, November 45 stories · 16 min
Models Lose the Plot, Laptops Read Weights
Today's stories test what AI can actually do: following long chains of updates, fixing security bugs, reading game weights, and working without production logs or unreliable arithmetic.
Picked from 66 new articles across 24 sources · arrow keys for other days
Today in 30 seconds
30s left
StateBench: frontier models fall to 13% accuracy or lower on the longest chained-update inputs; transformers never learned that level even when trained directly on it. story 1 ↓
Legora's six-person team builds simulated legal work, including a 10,000-file data room, because client confidentiality rules out production logs for testing agents. story 2 ↓
Simon Willison's 17GB local Qwen named three of four games from raw network weights, about an hour each, tempering sweeping readings of the original paper. story 3 ↓
CWE-bench v2: top models still fail 19% of fixes across 100 tasks; Gemini 4 Argon narrowly leads Claude Opus 5.5, while DeepSeek gets 57% for $0.04. story 4 ↓
DeepMind's Cheminformatics Skills put RDKit calculations under Antigravity's reasoning model; a demo went from literature search to proposed molecules in an hour, without benchmark numbers. story 5 ↓
1/5Hugging Face· Nov 3
StateBench finds frontier models lose track of long chains of updates
The Allen Institute for AI built a four-level test of following changing facts through long text. Frontier models fell to 13% or lower on the hardest long inputs; directly trained recurrent models went much further.
Why it matters · It separates architecture limits from training gaps, and shows that handling long inputs doesn't guarantee following chained updates, as code, logs and long agent sessions may require.
Know this first
Context from the editor, not the article · 28 sec
1 · Attention versus running memory
Transformers answer by attending back over earlier tokens through a fixed number of layers. Recurrent models instead update a fixed-size memory at every token, so information is carried forward step by step as they read.
2 · Long-context benchmarks
Tests of long inputs often hide a fact in a large document and ask for it back, or ask for simple tracking. Passing shows a model can locate information, not necessarily that it can combine many sequential changes.
3 · The swapped-cups problem
Tracking a ball under five shuffled cups is a classic hard case, called S₅ after the mathematical group of all rearrangements of five objects. Earlier theoretical work uses it to probe the limits of state tracking.
2/5Latent Space· Basil Chatha· Nov 2
How Legora tests legal AI agents without seeing customer data
Legora's CTO explains how the legal AI company evaluates agents it cannot observe: simulated legal projects, scored per practice area, plus cheap automated checks in production. He says models already handle about 60% of legal work well.
Why it matters · Any team shipping agents into regulated or confidential work may lose access to production logs; this is a concrete account of what replaces them and what remains blind.
Know this first
Context from the editor, not the article · 29 sec
1 · Traces and evals
A trace records each step of an agent run: prompts, tool calls and outputs. Evals are repeatable tests that score those runs. Agent teams usually mine real traces to find failures and turn them into test cases.
2 · Legal due diligence
Before an acquisition, a buyer's lawyers inspect the target's contracts and records, usually in a shared repository. They look for risks such as unusual pay terms or missing clauses, work traditionally done by junior staff at hourly rates.
3 · The billable hour
Law firms have long charged for time spent, so a tool that halves the hours can also halve revenue on that work. That incentive is the classic explanation for why earlier legal software struggled to win firms over.
3/5Simon Willison· Nov 3
A laptop model identifies games from raw neural network weights
Simon Willison reran a viral paper's weight-reading test on Qwen 3.8 27B, a model on his laptop. With no tools, it named three of four games from raw numbers, but he doubts the wider hype.
Why it matters · If a 17GB local model can do this with slow bookkeeping, the capability looks less like frontier magic and easier to test, which tempers the paper's most sweeping interpretations.
Know this first
Context from the editor, not the article · 27 sec
1 · What model weights are
A neural network is built from numbers called weights, set during training. They determine its behaviour, but no single number has an obvious meaning, so working out what a network does from them alone is hard.
2 · Policy networks
Reinforcement learning trains a network through trial and reward. A policy network takes the current game state as input and outputs the next move, so its first-layer weights connect directly to specific board positions.
3 · Running models locally
Open-weight models can be downloaded and run on personal hardware, typically compressed and much slower than hosted services. In a name like Qwen 3.8 27B, the suffix conventionally indicates about 27 billion parameters.
4/5Hugging Face· Nov 3
Open security benchmark finds top models still fail 19% of fixes
Collinear AI released 100 public tasks asking AI agents to find and fix security flaws in real open-source code. Of 11 models, Gemini 4 Argon led, solving 67.8% on a single try.
Why it matters · It offers a reproducible way to test models on your own defensive security work, and its cost data suggests the priciest model may not be the sensible default.
Know this first
Context from the editor, not the article · 29 sec
1 · What a CWE is
The Common Weakness Enumeration is a public catalog of software and hardware flaw types, such as SQL injection or buffer overflow. A CWE names a class of mistake; a specific bug in a specific product is usually tracked separately as a CVE.
2 · Pass@k scoring
Agents are non-deterministic, so benchmarks often run each task several times. Pass@1 estimates the chance one attempt succeeds. Pass@4 counts tasks solved in at least one of four attempts, showing what retries can reach.
3 · Why patch grading is hard
A security flaw can be fixed in many valid ways. Grading against one known patch rejects good alternatives, while a loose test can reward a fix that simply removes the feature. Good graders test behaviour instead.
5/5Google DeepMind· Nov 3
DeepMind gives its Antigravity agent deterministic chemistry tools
DeepMind released Cheminformatics Skills for its Antigravity agent surface: the model orchestrates and reasons while deterministic chemistry tools compute. A demo curated thousands of compounds, trained potency models and proposed new molecules in an hour.
Why it matters · It is a concrete template for making agents reliable in exacting domains: keep reasoning in the model, push every calculation into deterministic tools, and encode validation practice in the skill.
Know this first
Context from the editor, not the article · 23 sec
1 · What an agent skill is
A packaged bundle of instructions plus callable tools that teaches a general-purpose model a domain workflow, so it calls verified software instead of guessing answers from its own weights.
2 · Potency and binding
A drug works by physically binding a protein target. Potency measures how little compound is needed for an effect; it depends on how favourably the molecule fits the binding site.
3 · Why held-out splits matter
In chemistry machine learning, random train/test splits leak near-identical molecules across both sides, inflating scores. Splitting by core structure or by date tests whether a model generalizes to genuinely new compounds.
Open pretrained transformer, hybrid and Gated DeltaNet accuracy versus input length; soluble results are not reported.1 / 3
The Allen Institute for AI (Ai2) has released StateBench, a benchmark for 1: following how facts about entities change across a long document. Frontier models handle the easy kinds, but fall to 13% accuracy or lower on the longest examples of the hardest kind.
Why a new test
Reading code means following variables as they are reassigned. Following a mystery means knowing who holds the knife. Transformers, the dominant model architecture, have documented theoretical limits on long dependency chains. That helped motivate 2, a recurrent layer that carries a running memory forward as it reads, and hybrids that mix such layers with attention.
Existing tests focus on the hardest cases. Ai2 wanted to know which simpler kinds of tracking each architecture can handle too.
Four levels
R-trivial: a value never changes (let x = 4). Just recover it.
Aperiodic: values get overwritten (x = 4, later x = 2). Find the latest.
Soluble: updates build on the current value (x = x + 5), like a running score.
General finite: variables copy each other (x = y, or swaps), like tracking a ball under shuffled cups. One missed swap can ruin the answer.
The levels come from 3, a branch of algebra that breaks state-tracking problems into atomic primitives. To keep comparisons fair, Ai2 derives all four versions from one sequence. Variables, values and assignment positions match; only the assignment type changes.
Results
Frontier models. Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and GPT-6 Luna did well on the first two levels. Accuracy fell sharply with length on the last two. The authors suggest one likely factor: at harder levels, many more steps affect the final answer, leaving less room to miss one.
Open pretrained models. These included two 7-billion-parameter models, Olmo 3 (a transformer) and 4 (with Gated DeltaNet layers), plus smaller ones. Almost all did well on r-trivial, weakened on aperiodic as inputs grew, and stayed at chance on general finite. Format also mattered: small hybrid and Gated DeltaNet models scored near the ceiling on r-trivial problems in the style of 5, an existing long-context benchmark, but near chance on the same problems framed as spreadsheet edits.
Trained directly. Ai2 trained 275-million-parameter models from scratch on sequences of up to 256 assignments, then tested them on up to 64,000. Gated DeltaNet and hybrid models stayed almost perfect on the first three levels throughout. Transformers degraded with length and never learned general finite. The best Gated DeltaNet variant stayed near perfect on that level to about 10 times its training length.
How to read it
Ai2 concludes that hybrids' pretrained failures reflect training rather than architecture, while transformers hit a harder wall. Two caveats: these were small models trained on one task, and Ai2 also builds a hybrid model. StateBench also assumes a fixed set of variables, whereas real conversations keep introducing new entities.
The Allen Institute for AI (Ai2) has released StateBench, a benchmark for long-context 1: how well a language model keeps track of facts about entities as they change across a long document. Its headline result is that Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and GPT-6 Luna all handle the easier kinds of change, then fall to 13% accuracy or lower on the longest examples of the hardest kind.
The problem
State tracking sits under ordinary tasks. Reading code means following variables as they are reassigned. Reading a mystery means noticing that "John gave Mary the knife" changes who holds the knife. A harder version: John gives Mary the item in his backpack, and much earlier Edward put a knife in that backpack. Now the answer depends on a chain, and one missed link breaks it.
Theory gives a reason to expect architectures to differ here. Transformers have documented limits on long dependency chains. The post doesn't spell out the argument. The usual version, from complexity theory, treats a transformer as a parallel computation with fixed depth: its layer count stays the same however long the input gets, while composing a long chain of order-dependent steps appears to need depth that grows with length. The result rests on standard but unproven assumptions.
That argument helped motivate 2, a recurrent layer that carries a fixed-size memory forward and rewrites it as it reads, and hybrid models that combine such layers with attention. Ai2's complaint is that existing state-tracking tests concentrate on the hardest cases. As recurrent and hybrid designs spread, researchers also need to know which simpler, possibly more common kinds of tracking each one handles.
Four levels of difficulty
StateBench defines four levels, which the authors describe in terms of variable assignment:
R-trivial: immutable assignment.let x = 4 and the value never changes, like a birth date. The model only has to recover the original value.
Aperiodic: reassignment by value.x = 4, later x = 2. Only the latest write matters, as with a thermostat.
Soluble: self-referential updates.x = x + 5. Each update builds on the current value, as with a game score, so the model must carry changes forward.
General finite: assignment by reference.x = y, or swapping two variables, which creates dependencies between variables.
The magician's cups show why the last level is hard. As an illustration with three cups, put the ball under cup A. Swap A and B, and it is under B. Swap A and C, and it stays under B. Swap B and C, and it ends under C. Miss that last swap and you answer B, with no later step to correct the error. Tracking five objects this way is the 3 problem, and it belongs to the general finite level.
The levels aren't just convenient labels. They come from 4, a branch of algebra that breaks state-tracking problems into atomic primitives. Formally, these problems are word problems over 5: given a sequence of update operations, work out their combined effect. Each StateBench level corresponds to a class of problems built from a different set of primitives.
To keep the comparison fair, Ai2 starts from a general finite sequence and derives the three easier versions from it. All four use the same number of variables and distinct values, with assignments at the same positions. Only the type of assignment changes, so length and vocabulary can't explain differences in score.
Results
Open pretrained models. Ai2 started here because training data and context lengths are fully documented. The set included two 7-billion-parameter models, Olmo 3 (a transformer) and 6 (which adds Gated DeltaNet layers), plus smaller 275-million and 2.7-billion-parameter models. All were pretrained on the same data, and none saw contexts longer than 64,000 tokens; most were tested well beyond that. Scored on r-trivial, aperiodic and general finite problems, almost all did well on r-trivial, lost accuracy on aperiodic as inputs grew, and stayed at chance on general finite.
Format mattered too. The 275-million-parameter hybrid and Gated DeltaNet models scored near the ceiling on r-trivial problems written in the style of 7, an existing long-context benchmark, but near chance on the same problems framed as spreadsheet edits. That gap is one reason StateBench presents the same problem in several formats.
Frontier models. The four models read each sequence and generated the final value of every variable. All did well on r-trivial and aperiodic problems. On soluble and general finite problems, accuracy fell sharply with length. The authors suggest sensitivity as one likely factor: at the harder levels, many more steps can affect the final answer, so there is less room to miss an update and still be right.
Models trained directly on the task. This experiment asks whether failure comes from the architecture or from what pretraining teaches. Ai2 trained 275-million-parameter models from scratch: two transformer variants, two hybrids and two Gated DeltaNet variants with different memory-update constraints. Training sequences ran up to 256 assignments; tests ran to 64,000, or 250 times longer.
Gated DeltaNet and hybrid models generalized almost perfectly on r-trivial, aperiodic and soluble problems out to tens of thousands of assignments. Transformers learned the easier levels but degraded on longer sequences, and never learned general finite problems. At that hardest level, the Gated DeltaNet variant with the more flexible memory update went farthest, staying near perfect to about 10 times its longest training sequence, or roughly 2,560 assignments.
Ai2 reads this as two different failures. Pretrained hybrids and Gated DeltaNets fell short because of training, since the same architectures succeed when taught the task directly. Transformers failed even under these favorable conditions.
What it doesn't settle
Having the right architecture was not enough on its own: a hybrid pretrained on ordinary data still sat at chance on the hardest level. The direct-training result shows what these small models can learn on a dedicated task, not that a general-purpose model would acquire the skill. Ai2 also builds Olmo Hybrid, so readers should note its interest in the recurrent direction, though the benchmark and code are public.
The frontier result needs care as well. The post doesn't discuss the internal architectures of those models or say whether they could use tools, so it can't pin their failures on architecture.
The authors list further open questions. A new proof shows that Gated DeltaNet with a more limited memory update (non-negative 8, numbers describing how an update scales the memory) can solve all aperiodic tasks and some periodic ones, such as tracking whether an input has occurred an even or odd number of times. That holds under the paper's fixed-precision arithmetic with rounding. Whether this variant solves all soluble tasks is unknown, as are the effects of numerical precision and of adding attention layers. StateBench also assumes a fixed set of variables, while real conversations keep introducing new people and objects.
Remember
Finding a latest value and following a chain of updates are separate skills; models can have the first without the second.
Rewording the same problem as spreadsheet edits moved small models from near ceiling to near chance, so format belongs in evals.
A capable architecture isn't enough: pretrained hybrids stayed at chance on a level that directly trained hybrid-family models could learn.
Legora builds AI agents for lawyers, and it has a problem most agent teams don't: it can't read what its agent does. In a Latent Space interview, CTO Jacob Lauritzen explained how the company improves a product used by more than 1,800 customers anyway.
Testing without 1
Teams usually debug agents by reading traces, the step-by-step logs of each run. Almost everything Legora's users upload is confidential, and much of it belongs to the law firm's clients, so traces are off limits.
Instead, three engineers and three lawyers simulate the work. They build what Lauritzen calls 2: self-contained test worlds with a task and a defined good outcome. One is a 10,000-file 3, the document repository reviewed in an acquisition. The agent runs 4, the pre-deal risk check, and must write a report that catches the right red flags. Results go on a heat map of legal specialty against task, because new models often improve at some work and regress at others.
In production, cheap LLM checks estimate whether the agent solved the problem, and Legora compares that signal across 5 during rollouts. Failures get sorted into 10 to 20 known types. That shows what's going wrong, but not much about why.
Evals weren't always worth the effort. Until recently, every model simply got better. One release lifted citation performance from about 70% to about 99% with no work from Legora.
Where models fall short
Lauritzen estimates models already handle about 60% of legal work really well, leaving a cost and latency problem. Complex analysis and editing across million-token contracts still need more intelligence.
Legora has started 67 models, adapting publicly released models, for narrow tasks like legal research. It has no plans to train a frontier model. To counter hallucinations, everything the agent writes carries a citation that highlights a bounding box in the source PDF.
Few parts, many products
With about 100 engineers, Legora builds on a few primitives. The best known is 8, where each row is a document and each column a question. Drop in 10,000 employment contracts and ask for termination periods and severance terms. It first runs just 10 rows, so users can fix prompts before running the rest. Lists, a Notion-style database, can trigger agents, and agents can create and read them. Staff lawyers with technical backgrounds configure new products on these parts without code.
The billable hour
Faster lawyers earn less under hourly billing, which is why Lauritzen first doubted the idea. He argues competitive pressure changed that: US firms are moving routine reviews to flat fees and the money is shifting to M&A and litigation.
These are a vendor's claims. The 60% is an estimate, the citation metric isn't defined, and no eval results are published. How closely the simulations match real, unseen work remains unknown.
Legora builds AI agents for lawyers. Its CTO, Jacob Lauritzen, sat down with the Latent Space podcast to explain how the company develops a product whose real usage it mostly can't inspect. Legora began in a Stockholm basement as ChatGPT plus Swedish law and a search tool. Its agent now drafts and edits Word documents, reviews thousands of contracts at once, and coordinates work between lawyers, clients and other agents. It has more than 1,800 customers, passed $100 million in annual recurring revenue in April, and employs about 100 engineers.
The problem: no 1
Most teams improve agents by reading traces, the step-by-step logs of what an agent did. Legora can't. Almost everything users upload is confidential, and much of it belongs to the firm's clients, so the firm can't simply grant permission. Lauritzen calls this extremely frustrating: "I have no idea what people are actually doing."
For a long time, Legora also didn't invest heavily in evals. Models kept improving on their own, so building a better harness, the software around the model, paid off more than elaborate testing. In one case the agent wasn't citing properly, and the next model release moved that from about 70% to about 99% with no work from Legora. The heat map below exists because that free ride ended: Lauritzen says many releases now improve some tasks while regressing on others, more often than you'd expect.
What replaces them
Simulated projects. An evals team of three engineers and three lawyers tries to approximate what customers do. Because lawyers work in projects, the team builds what Lauritzen calls 2: simulated worlds with a task and a defined good outcome, borrowing the term from reinforcement learning. One is a 10,000-file 3, the document repository reviewed in an acquisition, where the agent runs 4, the pre-deal risk check.
The scoring is split into stages. One asks whether the agent found the right red flags. Another asks whether the report contains the right material, leaves out odd material, and has sound wording, sound analysis and suitable concision. Results land on a heat map of practice area (a subfield of law) against task, so a new model's gains and losses show up by type of work. The show notes link a separate write-up on running 2,000 parallel agent evals on Modal Sandboxes.
A pool of experts. The team draws on about 100 5: people who spent three or four years as big-law lawyers, often with a computer science degree. Lauritzen calls them the hardest people to hire and a force multiplier, because they use the product daily and talk to clients daily.
Coarse production signals. Online, cheap LLM checks estimate whether the agent solved the problem. Comparing that metric across 6, switches that expose a change to only some users, shows whether a rollout improved things or "catastrophically" declined.
Asked whether an LLM could paraphrase real traces into synthetic ones, Lauritzen described a spectrum. Reducing a trace to one of three values is safe; merely stripping names may not be. Legora takes the safest end. So it classifies failures into 10 to 20 known modes, and tags task type and practice area. That tells it which failures are common, but very little about why.
The same caution shapes infrastructure. Everything runs on Azure because early law-firm customers rejected AWS and Google, while Microsoft was already approved through 365. Lauritzen says that resistance has since faded. Legora also skips ElevenLabs because it keeps its 7, the contract listing vendors that may touch customer data, very short. That favors broad suppliers that are good enough across tasks over the best point solution.
Where better models still matter
Lauritzen estimates that models handle about 60% of legal work really well, making that portion a cost and latency problem. Complex analysis and editing across million-token contracts still need more intelligence, and he expects the amount of work to grow as capability does. Nine months ago, with Sonnet 4.6 and Opus 4.6, sub-agent delegation for fan-out retrieval queries was difficult; he says it now works extremely well.
Legora resisted a model picker because a lawyer shouldn't have to decide whether a contract is a "Sonnet-high or an Opus-mid task." It wanted to keep customers on the 8, the set of best available tradeoffs, here across quality, cost, latency, governance and regional availability. It eventually caved, though the picker covers only the agent's primary model. Tools inside the product still route subtasks to different models.
Some choices are task-specific. An early Gemini model does well at transcription with speaker attribution. For drafting, Lauritzen finds the models Fable and Opus too verbose and Astra more concise; the article doesn't describe Fable or Astra further. Legora has started 910 models, adapting publicly released weights, for narrow tasks like legal research, with no plans for its own frontier model. Against hallucination, everything the agent writes gets a citation that highlights a bounding box in the source PDF, "multiple layers down."
Few primitives, many products
Six months in, Lauritzen showed a slide of the 300 Spartans and declared Legora would stay at 20 engineers forever. At 100, he says it still feels too small, so the goal is shipping few things that feel like many. The primitives are projects that hold files, the agent, 11 and lists.
Tabular Review treats each document as a row and each question as a column. As an illustration, 10,000 employment contracts with columns for termination period, IP assignment clause and severance make 30,000 cells; the limit is about a million. It first runs 10 rows, so users can fix prompts cheaply before committing. The agent can also create a review itself, wait for the results, flag oddities such as an unusual CEO bonus, and draft a client memo. Cells cite their sources, so reviewers can check them quickly, which Lauritzen says beat every other format Legora tried.
Lists are Notion-style databases with triggers: agents can create and read them, and a change can start an agent. Intake, a Slack bot, turns a request into a ticket on a list, and the legal team works it inside Legora. Legal engineers configure new products on these parts without code, and even the fact-checking product from Wexler, an acquired startup, uses them. 12, embedded with clients, write production code for integrations and feed lessons back, still without touching client data.
The business argument, and what stays open
Lauritzen's original objection to the idea was the billable hour: faster lawyers earn less. He now argues competitive pressure changed that. US firms are moving small reviews to flat fees or doing them free, while money concentrates in M&A, litigation and client time. He rejects the idea that junior lawyers are finished, arguing good ones are worth more, though he sells to their employers. He also expects enterprises to reach Legora through tools like Claude Cowork, Anthropic's agent product, while law firms work inside it directly, and doubts labs will cut Legora off.
These are one executive's claims. The 60% is an estimate, the 70%-to-99% figure comes without a definition, and no eval results are published. The biggest open question is how well simulated worlds match work nobody at Legora can see. The transcript available here stops around minute 39, so later topics come from the episode summary.
Remember
When traces are forbidden, reducing failures to a small fixed taxonomy preserves privacy but loses most diagnostic detail.
Model upgrades can regress specific tasks, so per-domain heat maps reveal what one aggregate score would hide.
Running a bulk job on 10 rows first turns expensive fan-out work into a fast prompt-iteration loop.
A paper this week claimed that frontier language models can read and edit neural network 1 directly. Simon Willison tried part of it at home and found that a model on his laptop can pass one of its tests, though slowly.
The claim
"Sparks of 2", by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, coins its title term for models analyzing and modifying weights, the numbers that define a network's behaviour.
Its headline results use 3. Astra edited 0.016% of the weights of a small model, Qwen3.5-0.8B, raising its score on 4, a math benchmark, from 7.9% to 48.1%.
Willison argues the paper mixes two questions: what models can do with weights without tools, and what they can do once armed with code. spaCy author Matthew Honnibal called the conflation misleading. Sean Goedecke argued the Qwen fix followed the steps a human ML researcher would take. Willison focuses on the no-tools question.
The test
The authors trained four 5, which map a game state to a next move, on simplified 10x10 versions of Breakout, Pong, Pac-Man and Freeway. The test prompt contains one network's weights as text and asks which game it learned, with no code or calculator allowed. Apart from the weights, the prompts are identical.
The results
Willison ran the published prompts on Qwen 3.8 27B, a 17GB open model, through 6, a desktop app for running models locally. He had to raise the 7, the amount of text a model can hold at once, because the 8,192-token default failed with errors.
Game
Time
Qwen's answer
Stated confidence
Pac-Man
46m 12s
Correct
72%
Pong
1h 4m 54s
Correct
75%
Breakout
52m 25s
Correct
88%
Freeway
Not given
Wrong (Pac-Man-like)
Not given
The Pac-Man run used 43,506 8. Willison also ran the Breakout prompt on Kimi K3.1 Pro, a hosted model, which got it right in 9m 41s.
How it gets there
The reasoning traces are long stretches of bookkeeping. Qwen converts each weight's position into a board cell and input channel, then looks for unusually large values and asks what they imply, such as where the agent usually sits.
Willison's verdict
Willison expects an experienced person with a notepad and pencil could reach the same answers with enough effort. He calls the paper cool, but rejects commentary suggesting the age of source code is over.
These are single runs of four prompts, so they show the behaviour is reachable on a laptop, not how reliable it is.
A paper that circulated widely this week claimed that frontier language models can read and edit the 1 of neural networks directly. Simon Willison took one of its experiments and ran it on his laptop. Qwen 3.8 27B, a 17GB open model, correctly identified three of four games from nothing but a text dump of floating point numbers.
What the paper claims
"Sparks of 2: LLMs can natively read & write neural network weights" is by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. The authors call it a blog post; Willison calls it "paper-shaped". Weight-Space Programming is their term for a language model analyzing and changing a network's weights by reasoning about what each part computes, rather than by writing a training loop.
The abstract makes three claims for 3:
It can tell what a network was trained to do by looking at the raw weights.
Without environment feedback, it can write small game-playing networks from scratch that outplay networks trained for 100 million steps with 4 (Proximal Policy Optimization, a standard reinforcement learning algorithm).
Asked to improve the small model Qwen3.5-0.8B at math without 5, the signals normally used to adjust weights in training, it found a problem causing reasoning loops and edited 0.016% of the weights. Under the same sampling settings, its score on , a math benchmark, rose from 7.9% to 48.1%. Its score on , an instruction-following benchmark, tripled.
The authors say this works without letting Astra write training loops.
Two questions, not one
Willison argues the paper combines two topics that should be treated separately: what a model can do with raw weights without running anything, and how well a model armed with code can operate on other models. spaCy author Matthew Honnibal called the paper misleading for conflating them. Sean Goedecke argued that the Qwen3.5-0.8B fix took the same steps a human ML researcher would use.
Willison sets the code question aside and tests the no-tools one, which he finds more interesting.
Why reading weights is a real test
A trained network's weights are not a program in any readable sense. The authors made the problem small enough to attempt. They trained four 8, which map a game state to a next action, on simplified 10x10 grid versions of Breakout, Pong, Pac-Man and Freeway, a 1981 Atari 2600 game. Each network's weights were then written out as text in a prompt that asks:
Which game was it trained on? Work from the weights. You have no tools: no code execution, no calculator, nothing to run. This is a single response.
The prompts are in the authors' GitHub repository, and Willison checked that they differ only in the weights.
Running it locally
Willison used his LLM command-line tool, passing each prompt's URL through its fragment option, -f. Through the OpenRouter service, the hosted model Kimi K3.1 Pro identified Breakout after 9m 41s and 19,493 9, the text a model generates while thinking before it answers.
Qwen ran in 10, a desktop app for local models. It took two adjustments. The default 11, the amount of text the model can hold at once, was 8,192 tokens and failed with errors, so he loaded it with a longer one. He also raised the plugin's timeout using an environment variable.
Game
Time
Qwen's answer
Stated confidence
Pac-Man
46m 12s
Correct
72%
Pong
1h 4m 54s
Correct
75%
Breakout
52m 25s
Correct
88%
Freeway
Not given
Wrong: "Pac-Man-like"
Not given
The Pac-Man run produced 43,506 reasoning tokens, which works out to about 16 tokens per second by our arithmetic. The answers contained real detail. For Breakout, Qwen described a paddle on the bottom row moving left and right, with bricks or reward targets in the top rows. For Pong, it placed a vertical paddle on the right side intercepting a moving ball.
Willison wasn't surprised by the Freeway miss, reasoning that the game is more obscure than the other three. That is his guess, not something he tested.
What the reasoning looks like
Willison calls the traces "almost incomprehensible", but his sample shows a recognizable method. Qwen works out what each input to the network represents. One line reads "line382 input381: 381//5=76 (row7 col6), ch1".
Unpacked, that is ordinary index arithmetic. It treats the input as 5 channels per board cell, so input 381 divided by 5 gives cell 76 with remainder 1, meaning channel 1. On a board 10 cells wide, cell 76 is row 7, column 6. Having located a weight on the board, Qwen can ask whether large values cluster somewhere, and so reason about questions like "Agent position could be near bottom-right?"
That interpretation is ours, drawn from one short excerpt, but it fits Willison's assessment. He expects a sufficiently experienced person with a notepad and pencil could get there with enough effort. On this view, the model isn't perceiving the network's purpose directly. It is grinding through tedious analysis in text.
One loose end: Qwen's answers describe an 8x8 field while the article describes a 10x10 grid. Qwen also mentions wall borders, which could account for the gap, but the article doesn't address it.
What this doesn't show
The evidence here is modest. Willison reports one Qwen attempt per game, plus one Kimi run. Three correct answers can't establish a reliability rate, and the confidence figures are the model's own claims rather than calibrated probabilities. The article also describes no control, such as untrained weights, which could show how often the model produces a plausible game name anyway. These three games are also natural guesses for a grid-based arcade task, although the Freeway miss shows Qwen doesn't simply give the same answer every time.
Nor does the test touch the paper's more striking claims. Naming a game from a tiny policy network is far from writing a winning network from scratch or repairing a language model's math ability. Willison reproduced only the first kind of task.
His conclusion is deliberately restrained. He calls the paper cool, but says some of the wider commentary isn't justified, and he isn't ready to declare that source code is finished or that the future belongs to machines editing their own weights in incomprehensible ways.
Remember
The no-tools test is separate from code-assisted model editing, and evidence for one doesn't automatically support the other.
Qwen's traces show laborious position-by-position bookkeeping, which Willison thinks a skilled human with a pencil could replicate.
Long context and patience were practical requirements: Qwen's successful runs took 46 to 65 minutes, and the default 8,192-token context failed.
Pass@1 against cost per attempt for evaluated models, with the Pareto frontier marking the best trade-offs.
Collinear AI has released 1, a benchmark of AI agents' defensive security skills, with 100 public tasks anyone can run. Each task gives an agent a real open-source repository and a short instruction: find and fix a security weakness without breaking existing functionality.
The tasks span 91 CWEs, or entries in the 2, a catalog of software flaw types. With so few repeats, the authors argue, recognising a familiar pattern isn't enough. Collinear also keeps a 3 of unpublished tasks to track progress on work no model has seen.
How fixes are graded
The hard part of a benchmark like this is the grader. Security flaws can be fixed many ways, so comparing against one patch rejects good answers, while a loose test may reward a fix that just removes the feature.
CWE-bench uses 4, deterministic programs that test behaviour, so anyone can reproduce results. 5 helped curate these checks during development, but don't score runs. Collinear says it confirmed that the starter code fails, the reference fix passes, original functionality survives, and every graded behaviour can be inferred from the instruction.
It also checked that verifiers reject memorized solutions, including the known 6, so a model can't win by reciting a fix from its training data. The article doesn't explain how this works. Every task was also reviewed by hand.
Results
Eleven models were evaluated. 7 is the single-attempt success rate. Pass@4 appears to count tasks solved in at least one of four attempts, though the article doesn't define it.
Gemini 4 Argon: 67.8% pass@1, 81.0% pass@4, $3.92 per attempt
Claude Opus 5.5: 67.3% pass@1, 78.0% pass@4, $1.83 per attempt
DeepSeek V4.1 Flash: 57.0% pass@1, $0.04 per attempt
These three form the 8: no other evaluated model is both cheaper and better on pass@1. Opus adds ten points over DeepSeek for about 46 times the cost, and Gemini adds half a point for about twice Opus's cost.
At least one model solved 95 tasks, which Collinear takes as evidence they are solvable. Even so, the leader failed 19% of the set on every attempt.
Caveats
The half-point gap at the top is about two trials' worth of reward out of 400, and no confidence intervals are reported, so we wouldn't call it a real lead. The article also doesn't name the agent used for each model, which can affect score and cost, or compare results with v1. The vendor built this benchmark itself, though the public set, grader and harness are open to outside scrutiny. You can run it with 9, a tool for running packaged agent tasks in Docker.
Collinear AI has released 1, a benchmark of AI agents' defensive security skills, with a public set of 100 tasks anyone can run. The tasks cover 91 different CWEs, or entries in the 2, a catalog of software flaw types. Each task gives an agent a checkout of a real open-source repository and a short instruction. The agent must find and fix a security weakness without breaking existing functionality.
Collinear also keeps a 3, tasks not published and so unseen by any model, to keep tracking progress. Evaluating on it requires contacting the company.
Why the coverage matters
Collinear's reasoning is that a defender must anticipate every route an attacker could take and get it right every time. Spreading 100 tasks across 91 weakness types means there are few repeats of any one class, so, the authors argue, recognising a familiar pattern won't carry a model through.
Grading without a human or a model
Scoring is fully programmatic: each task has a verifier, a deterministic program that decides whether the fix works. That makes results reproducible and adds no judging cost at evaluation time. 4 did help curate the checks during development, pushing them towards being both hard and fair, but they do not score runs. The article doesn't describe how those panels work.
The difficult part is writing a verifier that doesn't accept bad fixes or reject good ones. Collinear says it checked each task against six questions:
Does the instruction read like a realistic request rather than an evaluation setting?
Can every graded behaviour be inferred from the instruction?
Does the verifier check for correct behaviour, not the structure of one particular fix?
Does the starter code fail and the reference fix pass?
Does the original functionality still work after the fix?
Do the 5 fail memorized solutions, such as the known 6?
Here is an illustrative example, not taken from the benchmark. For a SQL injection flaw, a behavioural verifier would send a malicious input and confirm it no longer works, then send a normal query and confirm it still returns the right rows. Parameterized queries and strict input validation could both pass, while deleting the query endpoint would fail.
The last question is the most striking, because it means pasting in the real project's published fix should not pass. That guards against a model reciting a patch from its training data. The article doesn't explain how the tasks achieve this. Every task was also reviewed by hand.
Results
Eleven models were evaluated. A run makes four attempts per task, or 400 trials, and the reported score is the mean reward across them. 7 is the single-attempt rate. The article doesn't define pass@4, but its figures fit the usual meaning: tasks solved in at least one of four attempts.
A rollout is one attempt at one task. These three form the 9, meaning no other evaluated model is both cheaper and better on pass@1. DeepSeek's entry carries a "(1017)" tag in the article, which it doesn't explain.
Dividing cost by pass rate, a rough calculation of ours, gives about $0.07 per passing attempt for DeepSeek, $2.72 for Opus and $5.78 for Gemini. Opus adds ten points over DeepSeek for roughly 46 times the cost. Gemini then adds half a point for about twice Opus's cost.
Two figures bound the difficulty. At least one model solved 95 of the tasks, which Collinear reads as evidence they are solvable. But the leader's 81% pass@4 means it failed 19% of the set on every attempt, and the five tasks no model solved remain open.
Running it
Each task is a self-contained package for 10, a tool for running agent tasks, made up of a Dockerized environment, an instruction and a verifier. Harbor can pull the benchmark directly from Hugging Face:
The -k 4 flag sets four attempts per task. The dataset is gated, so you must accept its conditions on the dataset page and store your Hugging Face token with git's credential helper:
11, whose weights are downloadable, appear on a leaderboard on the dataset page, which updates when someone opens a pull request on the model's repo with a results file. A separate leaderboard Space also includes API-only models such as Gemini, Claude and GPT.
The tasks are also tagged rl-environment, so the same environments and verifiers can supply rewards for training defensive agents.
What to be careful about
The half-point gap between Gemini and Opus is about two trials' worth of reward out of 400, and the article reports no confidence intervals, so we wouldn't treat pass@1 as separating them. The pass@4 gap of three tasks is also small.
The article's text leaves other things open. It doesn't name the agent used for each model, which can affect both score and cost. It gives no comparison with v1 and doesn't state the other eight models' scores. Public tasks can also leak into future training data, which is why the held-out set matters. Finally, this benchmark measures finding and fixing flaws, not exploiting them, and it comes from the company that built it, so outside review of the now-open tasks will be the real test of Collinear's quality claims.
Remember
Pass@4's gain over pass@1 suggests retries help: Gemini rises 13.2 points, though repeated attempts also multiply cost.
Rejecting the known upstream patch is an unusual anti-memorization check worth copying in your own evals, if it can be done fairly.
Spreading 100 tasks over 91 weakness types favours breadth, leaving little data on any single class.
Google DeepMind has released a set of 1 for 2, Google's agentic coding and research surface, and published a walkthrough in which an agent compresses a multi-day medicinal-chemistry loop into about an hour.
The problem: molecules are a data-wrangling nightmare
Chemistry has a huge public record, but the same molecule shows up in incompatible forms — an IUPAC name in a paper, a 3 (a generic template covering a family of compounds) in a patent, an 4 file in a vendor catalog. The authors note there's no margin for error: change one atom or bond and you've described a different compound, and a promising inhibitor becomes inert.
The mechanism: a division of labour
The skills split work explicitly. The language model does orchestration, database navigation and scientific reasoning; 5 chemistry tools do the actual molecular computation. Those tools are grounded in 6, the two-decade-old open-source cheminformatics toolkit, with connectors to 7 and 8, two curated public compound-and-bioactivity databases. Best practices are encoded into the agent's workflow rather than left to the model's judgment.
The worked example: rigidification
The demo targets a classic potency trick. Drug molecules joined by 9 are "floppy" — they twist into many 3D shapes, but only one shape binds the target. Giving up that freedom costs entropy, which weakens binding. Chemists therefore add bonds to lock the molecule in the binding shape in advance.
The case study is real: crizotinib, a lung-cancer kinase inhibitor with five major rotatable bonds, versus lorlatinib, a later drug cyclized into a 10 with effectively zero — and higher potency.
The agent pulled thousands of compounds from literature, stripped out buffers and solvents, documented sources, and flagged that the macrocycles are mostly predicted brain-penetrant — relevant because this cancer often metastasizes to the brain. An interactive viewer let potency be plotted against rotatable bonds, colored by macrocycle status; the most rigid compounds were the most potent.
It then ran 11 — comparing molecules differing by one change, here cyclization — confirming fewer rotatable bonds and generally higher potency. It benchmarked several model architectures and feature sets across four experimental readouts, using 12 (holding out whole chemical families) to estimate out-of-distribution performance, preferring chronological splits when dates exist. Finally it proposed new macrocycles with electron-pushing mechanism diagrams and synthesis routes, scored by its own trained models.
The honest caveat
The authors state the limits plainly: sparse data, inter-lab assay noise and narrow dynamic ranges cap predictive precision. Computational designs are testable hypotheses, not drugs; every one must still be synthesized and validated. No benchmark numbers accompany the demo — the claim is speed and support for hypotheses, not accuracy.
Google DeepMind has shipped 1 for Google 2, its agentic development-and-research surface, and published a technical walkthrough showing an agent run a drug-discovery loop that the authors say normally takes multiple days. In the demo it searches thousands of compounds, runs a standard medicinal-chemistry analysis, trains potency models, and proposes new molecules with synthesis routes — in about an hour.
Why chemistry data is unusually hard for agents
Chemistry is one of science's richest public records, spanning millions of structures across journal articles and vendor catalogs. But the authors' framing is that chemists still spend most of their time wrangling it, because the same molecule has many legitimate representations. It can appear as an IUPAC name in a paper (the systematic text naming convention), as a 3 in a patent (a generic template that covers a whole family of related compounds), or as a structure-data file — 4, a common text format carrying atoms, bonds and attached data — in a catalog. A system has to unify these and handle the many ways errors creep in.
What makes this different from ordinary data cleaning is the error tolerance. As the post puts it: alter a single atom or bond and you have described a different compound altogether, rendering a promising inhibitor or catalyst inert. There is no graceful degradation.
The mechanism: 5 tools under a reasoning model
The design choice worth noting is an explicit division of labour. The language model handles workflow orchestration, database navigation and scientific reasoning. Deterministic chemistry tools execute the molecular calculations — so the model never computes a molecular property by generating tokens that merely look like an answer.
Those tools are grounded in 6, the open-source cheminformatics toolkit built over two decades by a global community, plus connectors to 7 and 8, two large curated public databases of compounds and measured bioactivities. The skills also encode domain best practices directly into the agent's workflow, which matters later when the agent chooses how to validate a machine-learning model.
The chemistry being demonstrated: rigidification
The walkthrough centers on a classic technique for improving potency. Drug molecules often link several substructures through 9. Those bonds let the molecule twist into many three-dimensional conformations, making it "floppy" in solution. To bind its protein target, though, the molecule must lock into one specific active shape. Giving up that freedom of motion carries an entropic penalty, which weakens binding affinity and lowers potency.
The fix is to add bonds. Two substructures joined by a single rotatable bond can twist freely; add a second bond tying them together and the possible shapes collapse. Put that bond in the right place and the molecule is pre-locked into the binding shape — a strategy chemists describe as "paying the entropic cost in advance."
The case study is a real, well-known success. Crizotinib is a first-generation small-molecule tyrosine kinase inhibitor (a drug that blocks a signalling enzyme) used against certain non-small cell lung carcinomas; it has five major rotatable bonds. Lorlatinib is a third-generation inhibitor that resembles it but has been cyclized at one end into a 10 — a large ring — reducing the effective rotatable-bond count to zero, with higher potency among other effects.
What the agent actually did
Retrieval and curation. Asked to search the literature for these drugs and their analogues, the agent extracted molecules from the documents, associated each with its reported data, and within minutes retrieved thousands of compounds. It filtered out irrelevant molecules such as buffers and solvents, documented sources in a report, and added an initial analysis — including the observation that the macrocycles are largely predicted to be brain-penetrant. That is a substantive point, not decoration: this lung cancer frequently metastasizes to the brain, so crossing the blood-brain barrier is a design goal.
Interactive inspection. The agent launched a viewer where the scientist can browse every retrieved compound, click a molecule to see its metadata, and build scatter plots. Plotting cellular potency against rotatable bonds, coloured by macrocycle status, shows the pattern directly: macrocycles like lorlatinib have fewer rotatable bonds, and the highest-potency compounds are the most rigid.
Matched molecular pair analysis. MMPA isolates the effect of one discrete structural change by comparing pairs of molecules that differ only by that change. Here the pairs differed by whether or not they were cyclized, and the result matched expectation — cyclization reduces rotatable bonds while generally increasing potency. The post notes the agent had independently suggested this analysis.
Modelling. The agent built many potency models across varying feature sets and architectures, validated on a held-out set, benchmarked across four different experimental readouts. The validation choice is where encoded best practice shows: the skills steer the agent toward chronological splits — train on older compounds, predict newer ones — when dates are available, falling back to 11, which hold out entire chemical core structures, to better estimate generalization to genuinely new chemistry. Both are harder, more honest tests than a random split, which leaks near-duplicates between train and test.
Design. Finally the agent proposed new macrocycles, combining its own knowledge with the MMPA results and the potency models it had just trained. It explained its medicinal-chemistry rationale, generated curved-arrow electron-pushing diagrams showing reaction mechanisms, looked up similar reactions in public databases, suggested synthesis routes consistent with documented procedures, and predicted each design's potency with its own models. Outputs can be exported for industry-standard chemistry packages, or prepared for AlphaFold or molecular dynamics simulation.
Trade-offs and what's left open
The post is unusually direct about limits. Sparse data, inter-lab assay noise and narrow dynamic ranges cap the predictive precision of even the best models. Computational designs are testable hypotheses, not finished drugs — every proposed design must be synthesized and validated experimentally. The claim is that a scientist can generate more strongly supported hypotheses faster, not that the agent is right.
What the post does not provide is evidence in the form a sceptical engineer would want. There are no benchmark numbers: no accuracy figures for the potency models, no comparison against a chemist working unaided or against an agent without the skills, no count of how many proposed designs a medicinal chemist judged sound. The "multiple days to one hour" figure is the authors' characterization of a demonstration, not a measured study. The deeper question — whether constraining a model to deterministic domain tools plus encoded best practice is what makes agents reliable in high-stakes technical domains — is the interesting one here, and this post illustrates the pattern rather than testing it.
The skills are available now via the Antigravity app's Science plugin, and DeepMind is offering 10,000 scientists its top usage tier free for six months, subject to eligibility and capacity.
Remember
The design pattern: model does orchestration and reasoning, deterministic RDKit-based tools do every molecular calculation.
Encoded best practice matters — the skill pushes the agent toward chronological or scaffold splits over leaky random splits.
No benchmarks: no model accuracy, no baseline comparison, no expert judgement of the proposed designs. A demonstration, not a study.
