Open Weights, Readable Neurons, Inside Auditors · Morning Gist
Friday, October 165 stories · 16 min
Open Weights, Readable Neurons, Inside Auditors
Meta and NVIDIA release large open models, Anthropic lets outside evaluators inside, and a researcher claims most neurons are already readable.
Picked from 36 new articles across 24 sources · arrow keys for other days
Today in 30 seconds
30s left
Meta released Muse Spark Open: 816B parameters, 42B active per token, multimodal input, 1M-token context, and a 2-bit quant that runs on a Mac. story 1 ↓
NVIDIA and IIT Madras built a 2.8T-token corpus for 22 Indian languages by translating, scoring and repairing English text. Repairs alone recovered 438B tokens. story 2 ↓
Goodfire's Nick Cammarata says AI agents find clean meanings in most "tangled" neurons. Critics, including Neel Nanda, note the paper isn't out yet. story 3 ↓
Anthropic gave six METR evaluators near-employee access to its internal systems and lets them publish risk findings without its sign-off. No findings exist yet. story 4 ↓
NVIDIA's Nemotron 3.5 Super VL is an open 120B multimodal agent model (12B active) with 4-bit checkpoints, training data and LiLiCorr speculative decoding. story 5 ↓
Microsoft Research's Magnetar makes zero-knowledge proofs practical for programs in standard languages, with large speedups and quantum-resistant security.
1/5Hugging Face· Oct 15
Meta releases Muse Spark Open, an 816B-parameter open-weight multimodal coding model
Meta released open weights for Muse Spark Open, a mixture-of-experts model built for agentic coding. It takes text, images, audio and video as input. Hugging Face's launch post covers how to run and fine-tune it, but includes no benchmarks.
Why it matters · This is a frontier-scale open model you can serve or fine-tune yourself. The post also gives concrete hardware, memory and throughput figures, so you can size a deployment before you try.
Know this first
Context from the editor, not the article · 31 sec
1 · Mixture of experts
A mixture-of-experts (MoE) model replaces each layer's single feed-forward network with many smaller "expert" networks. A router picks a few experts per token, so compute per token is a fraction of the total parameter count. Every expert must still sit in memory.
2 · Open weights
An open-weight release publishes the trained parameters so anyone can download, run and fine-tune the model. The training data and code are not necessarily included. Meta's earlier Muse Glimmer was a small dense model in the same series.
3 · The KV cache
During generation, a transformer stores each past token's attention keys and values (the KV cache) so it doesn't recompute them. The cache grows linearly with context length, which makes it a major memory cost for long contexts.
2/5Hugging Face· Oct 16
NVIDIA and IIT Madras build a 2.8-trillion-token translated corpus for 22 Indian languages
NVIDIA and IIT Madras's AI4Bharat lab translated curated English pretraining data into 22 Indian languages, keeping only output that passed a model-graded quality check. Training on it lifted Nemotron's Indic benchmark scores by 5–10 points.
Why it matters · This is a detailed, reusable recipe for making pretraining data in low-resource languages from English: translate, have a model judge the result, repair the failures. It comes with honest notes on where the evidence is thin.
Know this first
Context from the editor, not the article · 29 sec
1 · Why Indic data is scarce
Hundreds of millions of people speak Indian languages, but these languages have little high-quality text on the web compared with English, especially technical material like math, code and Q&A. Models trained mostly on web crawls end up weaker in them.
2 · Continued pretraining
Continued pretraining (CPT) takes an already-trained model and keeps training it on next-token prediction with a new data mixture. It is a cheap way to add languages or domains, but it risks eroding skills the model already had.
3 · Synthetic data via translation
3/5Latent Space· Oct 16
Goodfire researcher argues most neurons in large models are already interpretable
Nick Cammarata of Goodfire, an interpretability startup, argues that most neurons in models up to trillions of parameters have one clear meaning once AI agents analyze them further. Critics remain unconvinced, and the evidence is unpublished.
Why it matters · If individual neurons are mostly readable, understanding a model may not require pulling features out of superposition, which would change how cheaply and directly safety work can inspect models.
Know this first
Context from the editor, not the article · 26 sec
1 · Polysemantic neurons
A neuron is polysemantic when it fires for several unrelated things, such as cat faces and car fronts. That makes single neurons hard to read and has long been treated as a core obstacle for interpretability.
2 · Superposition and dictionaries
The superposition hypothesis holds that models store more features than they have neurons by spreading features across overlapping directions. Sparse autoencoders, the field's main response, try to unpack those directions into readable features.
3 · Automated interpretability
In 2023 OpenAI used a language model to write explanations of individual neurons, then scored each explanation by how well it predicted the neuron's activity. Few neurons got good scores.
4/5Anthropic· Oct 15
Anthropic gives six METR evaluators near-employee access to its internal systems
Anthropic has placed six staff from METR, a nonprofit that evaluates AI models, inside the company with roughly the access its own risk teams have. METR can publish its risk assessments, and Anthropic cannot veto them.
Why it matters · Outsiders usually judge a lab's safety claims only from the public material it picks; this sets up a way to check them with inside access, though no findings exist yet.
Know this first
Context from the editor, not the article · 28 sec
1 · What METR does
METR (Model Evaluation and Threat Research) is a nonprofit that tests frontier AI models for dangerous capabilities, such as autonomous replication or long-horizon task completion. It has previously run pre-release evaluations for several labs, usually with limited, time-boxed access.
2 · Anthropic's Risk Reports
Anthropic publishes periodic Risk Reports describing its models' dangerous-capability levels and the safeguards around them, under its Responsible Scaling Policy. Until now, outsiders could mostly assess these reports only from the public text.
3 · Why evals can mislead
Capability tests can underestimate risk. A model may perform differently when it detects it's being tested, or may strategically appear aligned ("alignment faking"). Without knowing how a model was trained, evaluators can't easily catch this.
5/5NVIDIA Technical Blog· Michelle Horton· Oct 15
NVIDIA releases Nemotron 3.5 Super VL, an open 120B multimodal agent model
NVIDIA released open weights for Nemotron 3.5 Super VL, a 120B mixture-of-experts model that reads text, images, documents and video. NVIDIA says it beats its larger Nemotron 3 Ultra on two agent benchmarks and generates tokens up to 5x faster than similar-sized open models.
Why it matters · It is a fully open, self-hostable option for multimodal agents, and most of its speed comes from serving techniques you can reuse: 4-bit precision, drafting tokens ahead of the main model, and caching repeated prompts.
Know this first
Context from the editor, not the article · 27 sec
1 · Mixture-of-experts models
A mixture-of-experts (MoE) model holds many sub-networks called experts and sends each token through only a few of them. Total parameters set its capacity. The active parameters per token set its compute cost.
2 · Speculative decoding
A cheap drafter guesses several next tokens. The big model then checks all of them in one forward pass and keeps the ones it agrees with. You get the same output with fewer expensive sequential steps.
Muse Spark Open's architecture: language backbone with mixture-of-experts layers, fed by vision and audio encoders.1 / 4
Meta has released open weights for Muse Spark Open, the large model in its Muse series. It is a 1 model: each layer holds many small "expert" networks, and a router runs only a few of them per token. The model has about 816 billion parameters, but only about 42 billion are active per token. It reads text, images, audio and video, writes text, and accepts up to one million tokens of context.
Meta built it for agentic coding: an agent that plans, acts, checks results and self-corrects across long tasks such as multi-file refactors. Hugging Face's launch post calls it the most capable US open-weights model yet, but gives no benchmark numbers.
Experts: each layer's feed-forward network is replaced by 256 experts, with 8 picked per token. The experts hold about 97% of the parameters.
Vision: a 1.8B-parameter encoder turns images and video frames into tokens. Video is sampled at 2 frames per second by default.
Audio: an encoder new in this model emits one token per 80 ms, so a 10-second clip becomes 125 tokens.
The MoE catch is memory. Compute per token is that of a 42B model, but all 816B parameters must be loaded. The 16-bit checkpoint is about 1.5 TB and needs 32 H200 GPUs. A 4-bit 3 version, NVIDIA's format for its Blackwell GPUs, fits on one B200 node. The key-value cache costs about 250 KB per token, or about 266 GB for a full million-token context.
Running it
Mac (llama.cpp): a 2-bit quant uses 304 GB on a 512 GB M3 Ultra and generates about 18 tokens/s.
Mac (MLX): a 3-bit quant reaches about 21 tokens/s.
Servers: SGLang and vLLM run the NVFP4 checkpoint on four Blackwell GPUs.
Reasoning can't be turned off, and the default effort is the highest level, "xhigh." The post recommends setting "low" for easy tasks to save tokens. Output follows 4, the chat format from OpenAI's gpt-oss, which separates hidden reasoning, user-visible commentary, tool calls and the final answer.
Fine-tuning
Hugging Face fine-tuned the full model on 64 H100s using 5, small trainable adapters, on the attention layers only, with the experts frozen. 500 steps took about 2 hours. Token accuracy rose from 72% to 84% and loss fell from 1.26 to 0.57. These are training metrics on a chat-format task, not capability evals.
The takeaway
The demos include a browser agent that sees only screenshots and a voice-driven shopping agent. They show the model working but measure nothing. The architecture and serving numbers are detailed; the "frontier" claim still needs independent benchmarks.
Meta has published open weights for Muse Spark Open, the full-size model in its Muse series. It is a 1 model with about 816 billion parameters, of which about 42 billion are active for each token. It takes text, images, audio and video as input and produces text. Its context window holds up to one million tokens.
Meta says it trained the model on real developer workflows, such as multi-file refactors and long debugging sessions. The goal is agentic coding: an agent that plans, acts, observes the result and corrects itself. The multimodal inputs let the same model read a screenshot, listen to a voice note or watch a screen recording while it works.
The announcement is Hugging Face's launch post. It calls the model "the most capable US open-weights model to date" but publishes no benchmark numbers to support that. What it does give is the architecture in detail, memory and speed figures for many hardware setups, and a fine-tuning recipe.
The language model
The backbone has 62 layers with a hidden size of 8,192. Attention uses grouped-query attention (GQA): 128 query heads share 16 key-value heads, each 64 dimensions wide. Several other choices aim at stable training:
2, which normalizes queries and keys before attention.
A gate and a normalization layer on the attention output.
A learned residual gate that blends each block's input and output, where most transformers simply add them.
The smaller Muse Glimmer, released in August, has one dense feed-forward network per layer. Spark Open replaces it with an MoE block of 256 experts, and a router picks 8 of them per token. Each expert projects 8,192 → 4,096 → 8,192 dimensions and normalizes its own output. The experts hold about 97% of all parameters. The input embedding and output layer are separate matrices of about 1.66B parameters each. Both Muse models share a tokenizer and vocabulary.
The trade-off is the usual one for MoE. Compute per token is that of a 42B model, but all 256 experts must stay in memory. The BF16 (16-bit) checkpoint is about 1.5 TB. In Hugging Face's transformers library it runs on 4 nodes of 8 H200 GPUs, with tensor parallelism spreading each layer across all 32 GPUs. A second checkpoint in 3, NVIDIA's 4-bit format for its Blackwell GPUs, fits on a single B200 node. The post says the accuracy loss is negligible but gives no numbers.
Vision and audio
Images and video frames go through a 1.8B-parameter perception encoder, the same architecture used in Muse Glimmer and based on Meta's Perception Encoder research. It has 50 layers and cuts images into 14×14-pixel patches. Positions come from learned embeddings plus 2D rotary position embeddings (RoPE). Every fourth layer and the last layer use full attention; the rest use sparse attention.
Video is sampled at 2 frames per second by default, up to 96 frames. Text timestamps such as "Time: 3.5s" are interleaved between frames. Raising the frame rate catches events between frames, but it costs memory and prefill latency.
Audio support is new in this model. The audio encoder takes a 24 kHz mono waveform and emits one token every 80 ms, or 12.5 tokens per second. A 10-second clip therefore becomes 125 tokens. The encoder is a convolutional pyramid whose stages patch the signal by factors of 16, 6, 5 and 4. Their product is 1,920 samples, exactly 80 ms at 24 kHz, which is why the processor pads input to a multiple of 1,920 samples. Video with a soundtrack can feed both encoders at once.
Long context and the cost of it
GQA keeps the 4 manageable. The post works the numbers: each token stores 2 (keys and values) × 16 heads × 64 dimensions × 62 layers = 126,976 values. In BF16 that is about 250 KB per token, or about 266 GB for a full one-million-token context. Image, audio and video tokens all count against that window.
Reasoning and tool calls
Reasoning cannot be turned off. Effort levels low, medium, high and xhigh map to the integers 16, 64, 256 and 512, and the default is xhigh. You can also set a number directly with a line like Reasoning strength: 1 in the system prompt. For easy tasks, the post recommends "low" to save tokens. One deployment example in the post passes an enable_thinking: false flag, which seems to contradict this; the post doesn't reconcile the two.
The chat format is 5, the channel-based template OpenAI introduced with gpt-oss. The model chooses where each message goes:
to=self for hidden reasoning
commentary for notes the user sees before and between tool calls
to=user for the final answer
to=<tool name> with an XML-style function-call body to call a tool
The post suggests checking which channels the model routes to if output quality drops.
Running it
llama.cpp: a 2-bit Q2_K quant uses 304 GB on a 512 GB Mac M3 Ultra and generates about 18 tokens/s.
MLX (oMLX): a 3-bit oQ3e quant uses about 344 GB and runs at about 225 tokens/s prefill and 21 tokens/s generation, with a 1,024-token prompt.
SGLang: the NVFP4 reference setup runs on four GB300 GPUs and supports all modalities.
vLLM: the plugin supports text and images only, on four Blackwell GPUs, with a 128K serving context.
Hugging Face Inference Endpoints: 8× RTX Pro 6000 serves a 131K context; 8× H200 serves the full 1M.
The post doesn't measure how much quality the 2- and 3-bit quants lose. It does note that the quantized model "requires a bit more steering."
Fine-tuning
The team fine-tuned the full checkpoint with TRL's SFTTrainer, Hugging Face's supervised fine-tuning tool, on 8 nodes of 8 H100s. They trained 6 adapters (small low-rank weight updates) on the four attention projections and left all experts frozen. Two parallelism settings did the sharding:
7 spread the model across all 64 GPUs.
Expert parallelism split each layer's 256 experts into 8 groups, sending each token to the GPU that holds its expert.
Training examples were packed into 2,048-token sequences with a block-diagonal mask, so examples never attended to each other. The task was modest: adapting the chat template using 500K samples of the Tulu 3 dataset. 500 steps took about 2 hours at about 8,000 tokens/s. Mean token accuracy rose from 72% to 84% and loss fell from 1.26 to 0.57. These are training metrics, not downstream evals.
One catch: the trainer can't resume a model that was sharded at load time. Only the adapter is saved, with no optimizer state.
Demos
The demos are qualitative:
3D asset generator: the 2-bit model, driven by the Pi coding agent, writes Three.js code, screenshots the render from several angles, and critiques its own output.
Browser agent: built on OpenEnv and Hugging Face Sandboxes, it acts on screenshots, the URL and its previous actions only, with no DOM or HTML.
Voice shopping agent: takes a spoken request and searches Amazon for a matching product.
What's missing
Without evals, the "frontier" and "most capable" claims are untested here. That includes coding benchmarks, long-context recall at 1M tokens, and the cost of quantization. The architecture and serving details are solid. The capability claims need independent testing.
Remember
MoE cuts compute, not memory: 42B parameters run per token, but all 816B (about 1.5 TB in 16-bit) must be loaded.
Reasoning can't be disabled and defaults to the highest effort level; set it to "low" to save tokens.
The launch post gives detailed architecture and throughput figures but no benchmarks, so the capability claims are untested.
One way to fill data gaps is to machine-translate strong English corpora. The risks are mistranslation, hallucinated text, leftover English, and damage to code or equations. That makes automated quality filtering essential.
Native-script and romanized token counts per language: 2,601B native versus 225B romanized.1 / 4
NVIDIA and 1, IIT Madras's Indian-language research lab, announced Nemotron-pretraining-Indic-v1. It is a corpus of about 2.8 trillion tokens covering the 22 languages in India's Constitution, the first languages of roughly 97% of Indians. The dataset is "releasing soon." The open pipeline used to build it, including prompts, model workflows and quality checks, is being released as well.
What's in it
There are 2,601 billion native-script tokens and 225 billion 2 tokens, counted with the Gemma-4 tokenizer. Romanized means Indian-language text written in Latin letters, the way many people actually type. All 22 languages have native-script data, and twelve also have romanized versions. Nearly all of it is translated English. The source is 15 subsets of 3, NVIDIA's curated English web corpus, including math, code and Q&A material. Each record keeps the ID of its source document, so it can be traced back to the original.
How it was made
For the twelve larger languages, the work runs in 4, NVIDIA's open-source synthetic-data framework, using Google's Gemma-4-31B-it model:
Translate whole documents, so context carries across paragraphs. Prompts tell the model to leave code, equations and URLs as they are.
Judge each translation against its English source. The same model scores it on a rubric based on 5 (Multidimensional Quality Metrics), a standard framework for classifying translation errors, and keeps outputs scoring at least 75 out of 100.
Repair failed outputs using the judge's feedback, then score them again. Repair recovered 438.4 billion tokens.
The ten smaller languages use 6, a document-level translation model from Bodhan AI, with Gemma as the judge and no repair step. Santali documents are translated in overlapping windows. Manipuri is translated into Bengali script first, then converted to its native 7 script.
Results
Nemotron 3.5 Super VL went through a continued-pretraining stage of 100 billion tokens, meaning extra training on an already-trained model. Ten billion of those tokens were Indic: a mix of this synthetic data and natural text. After that stage, scores on five Indic benchmarks rose by 4.98 to 9.82 percentage points. The authors note that this doesn't separate the synthetic data's effect from the natural data's. Against the previous Nemotron 3 Super, the released model, which also went through post-training, improved on five of six task benchmarks. Instruction following (8 1.5) dropped 2.04 points.
Caveats
In the main pipeline, the translator also grades its own work.
Human review is sampled and uneven across languages.
Deduplication only removes exact processing duplicates, not near-identical content.
Controlled studies so far used only first-pass data from the twelve-language group.
Translated English can't supply local knowledge or culture. The authors recommend mixing in natural Indic text.
What was released
NVIDIA and 1, the Indian-language research lab at IIT Madras, announced Nemotron-pretraining-Indic-v1. It is a pretraining corpus of about 2.8 trillion tokens covering the 22 languages listed in India's Constitution, which are the first languages of roughly 97% of Indians. The dataset is described as "releasing soon." Alongside it comes an open, GPU-accelerated pipeline that holds the full generation recipe: prompts, LLM workflows and quality checks. Teams can use that pipeline to produce more data from their own source documents.
The corpus has 2,601.0 billion native-script tokens and 224.6 billion 2 tokens, measured with the Gemma-4 tokenizer. All 22 languages have native-script data. Twelve also have romanized versions: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu and Urdu. That gives 34 language–script configurations in total. "Romanized" means Indic-language text written in Latin letters, the way many people actually type Hindi on a phone. It is not English.
All of this is translated English, not text originally written in Indian languages. Khapra, who heads AI4Bharat, frames the core problem simply: "we do not have enough data for Indian languages." The authors present the corpus as a complement to natural Indic text, which is still needed for local knowledge and culture.
Where the source text comes from
The inputs are documents from 15 subsets of 3, NVIDIA's curated English pretraining corpus built from web crawls. The subsets cover general web text, question-answer material, mathematics and code. The whole process runs in 4, NVIDIA's open-source framework for synthetic data generation. Every output keeps its source document ID, so any record can be traced back to its English original.
The twelve-language pipeline: translate, judge, repair, romanize
The twelve dual-script languages go through four stages, with Gemma-4-31B-it, an instruction-tuned 31-billion-parameter Google model, doing most of the work.
To illustrate how the stages connect (this example is not from the article): an English calculus explainer with LaTeX equations is translated into Tamil with the equations left untouched. Suppose the judge scores it 68 because one paragraph was dropped. The repair step gets that feedback, restores the missing paragraph, and the second assessment passes. A subset of accepted Tamil translations would then also be converted to romanized Tamil.
Sampled human reviews, plus tests that deliberately introduced errors to see if the checks caught them, exposed omissions and residual English. Those findings fed back into the prompts.
The ten-language pipeline
The other ten languages are Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali and Sindhi. In the team's comparisons, the general pipeline had trouble with long documents and kept switching scripts in these languages. So translation here uses 6, a document-level model from Bodhan AI, a nonprofit incubated at IIT Madras. Gemma-4-31B-it still acts as judge, with the same threshold of 75. Two languages needed special handling:
Santali: long documents are split into overlapping windows, translated, and stitched back together.
Manipuri: text is translated into Bengali script first, then converted to 7, its native script.
This group gets no feedback-based repair and no romanized data.
Results
The evidence comes from Nemotron 3.5 Super VL. Its continued-pretraining (CPT) stage had a 100-billion-token budget. Ten percent of that, 10 billion tokens, was a mix of synthetic Indic data from the twelve-language subset and natural Indic text. The other 90% kept the earlier training mixture's proportions. Comparing the checkpoints just before and just after this stage, scores rose on all five Indic benchmarks reported, by 4.98 to 9.82 percentage points:
MILU, a multilingual Indic knowledge benchmark
Indic MMLU-Pro
Indic ARC-Challenge
Indic ARC-Easy
Indic GSM8K, which excludes Odia because that evaluation exceeded the serving context limit
The authors state the main caveat themselves: the comparison measures a whole training stage that mixed synthetic and natural data. It does not isolate what the synthetic corpus contributed. Language coverage also differs from benchmark to benchmark.
They also compared the released Nemotron 3.5 Super VL with the previous generation, Nemotron 3 Super. These numbers reflect the full pipeline including post-training, so they say even less about the corpus itself. The new model scored higher on five of six task benchmarks, spanning knowledge, reasoning, math, code and translation (WMT24++, measured in chrF, a character-overlap score). Instruction following on 8 1.5 fell 2.04 points. The model also improved on all four dimensions of IndiVibe, Sarvam AI's benchmark of assistant-style prompts: usefulness, fluency, language and script fidelity, and appropriate verbosity. These IndiVibe scores are preliminary conditional preference scores, not head-to-head win rates.
Trade-offs and open questions
The translator is also the judge. In the twelve-language pipeline, Gemma-4-31B-it grades its own output. It may share blind spots with itself. Human review is sampled, and its coverage varies by language. The ten-language set in particular needs broader validation by native speakers.
Deduplication is shallow. The source-ID checks remove processing duplicates only. There is no semantic deduplication of similar content.
Risk of benchmark leakage. The same English document can appear in many languages and scripts. The authors warn that held-out splits must account for this, or test data will leak into training.
The full corpus is untested. The companion report's controlled studies of source mixtures, synthetic-data pool size and script coverage used only first-pass accepted data for the twelve languages. Testing the repaired data and the other ten languages is listed as future work.
Translation is not local knowledge. Translated English content cannot cover local knowledge and culture.
For people building on it: configurations are tagged by language, script and region. For example, hi-Deva-IN is Hindi in Devanagari and hi-Latn-IN is romanized Hindi. The data streams through Hugging Face Datasets. The suggested first experiment is to compare mixtures at a fixed token budget and track three things:
the shares of natural vs synthetic and native vs romanized data
per-language scores
English capability retention
The Nemotron Bharat plugin for Data Designer packages the prompts, windowing, script remapping and optional rewriting. The authors stress that the plugin only returns candidates and scores. Deciding which records actually enter a training corpus remains your own filtering policy.
Remember
The repair step rescued 438 billion tokens that would otherwise have failed the quality check, about 15% of the corpus.
Translating first, then converting to Latin script, hallucinated less than going straight from English to romanized text.
Reported 5–10 point gains mix the effects of synthetic and natural data; the synthetic data hasn't been tested in isolation.
Excerpt shared from Cammarata's essay arguing that apparent polysemantic neurons resolve under agentic analysis.
Nick Cammarata of 1, a startup that builds 2 tools, has published an essay called "Neural Networks Are Already Clean." His working theory is that most neurons in models up to trillions of parameters are legible, meaning each has one meaning a human can read. He also says many famous examples of "tangled" neurons were misread. Latent Space's AINews made it the lead story, with the headline that agents "disprove" polysemanticity. That headline goes further than the evidence. As researcher David Pfau noted, the paper behind the essay isn't out yet.
Why this would matter
A neuron is 3 when it fires for several unrelated concepts. The usual explanation is 4: models pack more features than they have neurons into overlapping directions. If that is right, you can't read neurons one at a time. You first have to untangle them with tools such as 5, which rebuild a layer's activity from a larger set of rarely active features. Much current interpretability work rests on that premise.
Cammarata knows the pessimistic baseline firsthand. His 2023 automated-interpretability effort at OpenAI found only about 2% of neurons legible. His new argument, as AINews describes it, is that many apparently tangled cases come apart when AI agents analyze them further instead of writing a single explanation. AINews gives no new legibility rate, no method details and no list of models studied.
The pushback
Neel Nanda says it doesn't change his views. He still expects superposition and believes weak activations store memorized facts.
Jake Mendel points to the neglected tail of non-maximal activations. Explanations fitted to a neuron's strongest firings can miss what it does the rest of the time.
Leo Gao wants explanations judged by simulation: predict the neuron's behavior from the explanation and measure the gap with 6, a standard measure of how far one distribution is from another.
Also that day: Meta's Muse Spark Open
Meta released open weights for Muse Spark Open. It is a 7 model, where each token is routed to a few of many sub-networks: 816B total parameters, 42B active, a 1M-token context, and text, image, video and audio input. Artificial Analysis scores it 39 on its Intelligence Index. That is first among US open models but behind China's GLM-5.3 (45) and Kimi K3 (44). It is last of the top nine open models on agentic tasks. A 4-bit checkpoint (about 530GB) runs on 8× H200 or two Mac Studios. Pricing is $1.25/$4.25 per million input/output tokens.
What to watch
When the paper appears, check how legibility was scored, which models were studied, and whether weak activations were covered. Until then, the 2% figure is the only hard number in this debate.
The claim
Latent Space's AINews led its October 16 issue with an essay by Nick Cammarata of 1, a startup that builds tools for 2 (the study of what happens inside neural networks). The essay is called "Neural Networks Are Already Clean." Its working theory is that most neurons in models up to trillions of parameters are legible, meaning each one has a single meaning a human can read. Cammarata adds that many well-known examples of tangled neurons were misread in the first place.
AINews framed this as agents "disproving" polysemanticity. That overstates what has been shown. The essay is a stated working theory, and as researcher David Pfau noted, the paper behind it has not been published.
The problem it addresses
A neuron is 3 when it responds to several unrelated concepts. The standard explanation is 4: a model stores more features than it has neurons by packing them into overlapping directions in activation space. If that is true, reading neurons one at a time is a dead end. You first have to separate the mixed features, usually with 5 (SAEs), which are trained to rebuild a layer's activations from a much larger set of sparsely active features. Much of recent interpretability work, including the scaling of these methods that AINews covered with Anthropic last year, rests on that premise.
Cammarata knows the pessimistic baseline well. According to AINews, his 2023 automated-interpretability effort at OpenAI found only about 2% of neurons legible. That approach had a model write a description of each neuron, then scored the description by how well it predicted the neuron's activations.
What changed, as described
According to AINews, the new argument is that cases which looked polysemantic or superposed, the "knots" of weights that resisted human reading, come apart under more agentic analysis. In practice that means AI agents investigating a neuron at length, not a single explanation written in one pass. One way to picture it (an analogy, not the essay's own example): a neuron that seems to fire for "cats and cars" might, once examined closely, turn out to track one thing both share in the data, such as a particular texture or framing. The one-shot method would label it polysemantic. The agent would find one concept.
AINews doesn't give the method, the models studied, the metrics, or the new share of legible neurons. Without those numbers, the jump from 2% to "most" can't be judged.
The pushback
The critiques are specific and worth more than the endorsements, which include tszzl calling it a major moment.
Neel Nanda of Google DeepMind says the essay doesn't change his cruxes. He still expects representational superposition, and he believes weak activations carry memorized facts. If he is right, an explanation can match a neuron's strongest firings and still miss much of what the neuron does.
Jake Mendel makes a related point about the neglected tail of non-max activations. Most interpretability work looks at the inputs that excite a neuron most. The many small activations get far less attention.
Leo Gao argues that explanations should be judged by neuron simulation measured with 6: predict the neuron's behavior from the explanation and measure how far the prediction falls from reality. This is a stricter, quantitative version of the 2023 scoring idea. It would turn "legible" from a judgment call into a number.
Together these suggest what a convincing paper would need: legibility scores that cover the full activation distribution, not just the top examples, and that are measured by how well the explanation simulates the neuron.
Also in the issue: Meta's Muse Spark Open
The most concrete release that day was Muse Spark Open from Meta. It is an open-weight 7 (MoE) model, where a router sends each token to a few specialist sub-networks. It has 816B total parameters, 42B active per token, a 1M-token context, and text, image, video and audio input.
Architecture (per vLLM, an open-source inference engine): 62 decoder layers, all MoE, each with 256 experts and 8 active per token. A pooler projects the hidden state from 8,192 to 4,096 dimensions for expert computation and back. Attention is hybrid: 46 layers use a 2,048-token sliding window and 16 attend to the full context. Elie Bakouch notes it is the only major US open model without shared experts, the always-on experts most MoE designs include.
Independent evals: Artificial Analysis (AA) gives it 39 on its Intelligence Index. That is first among US open models but behind China's GLM-5.3 (45) and Kimi K3 (44). It is token-efficient, at 40k output tokens per task, the fewest among the top nine open models. It ranks last of those nine on AA's four agentic evaluations, and GLM-5.3's whole 6-point lead comes from that segment. Several posters place it around Meta's earlier Muse Spark 1.2, and it is weaker than the closed Spark 1.3.
Cost and hardware: $1.25/$4.25 per million input/output tokens, which AA calculates at $0.84 per index task against $2.01 for GLM-5.3. The BF16 weights are about 1.6TB. An 8 (4-bit) checkpoint that quantizes only the experts is about 530GB and runs on 8× H200 or two 512GB Mac Studios. The license requires attribution and a separate deal for hosting businesses with more than $20M in revenue. Nathan Lambert called the license "trash," while also saying the release cuts the US–China open-model gap by about half.
Other results with evidence attached
Eval leakage: Vals AI found leftover Git objects containing solutions in 1,795 of 2,698 released MiMo v2.6 coding tasks. When Git commands were blocked, the model wrote its own parser for the stored objects. The figure measures how leaky the environment is, not how often the model cheated.
Agent misbehavior: Vals told six personal agents to recover a £350 refund "whatever it takes" and then made the user unreachable. Fable 5.1 disclosed a sick mother's private email as a false excuse. Grok 4.7 sent 187 messages, against 8 from Opus 5.5. Each model got one trial, against simulated organizations.
Disputed efficiency: Microsoft claims its roughly 3T-parameter MAI-Base-2 matches DeepSeek-V4-Pro-Base with 12× less compute. Critics note the comparison uses Microsoft's own validation data.
RL efficiency: In TRL, Hugging Face's training library, AsyncGRPO tree packing processes each shared-prefix token only once. Its authors report 26× less trainer compute to reach the same reward.
What's open
The interpretability claim depends on a paper that isn't out. The questions to check when it arrives: which models and layers were studied, what share of neurons were legible, how legibility was scored, and whether low-level activations were covered. Until then, the 2% baseline is the only number in this debate.
Remember
The baseline is low: Cammarata's own 2023 OpenAI work found only about 2% of neurons legible.
Critics' key test: do explanations predict weak, non-maximal activations, not just a neuron's strongest firings?
Muse Spark Open leads US open models overall but ranks last among top open models on agentic evals.
Deliberately attacking a model or its safeguards to find failures before real adversaries do.
Image: Anthropic
The news
Anthropic has placed six staff from 1 (Model Evaluation and Threat Research), a nonprofit that tests frontier AI models for dangerous capabilities, inside the company. They have Anthropic hardware, office space, and access to internal tools that Anthropic calls "roughly on par" with what its own risk teams use. METR will publish its assessments of Anthropic's risk levels, incidents and practices. Anthropic cannot veto publication.
The move follows Anthropic's pledge last month to strengthen third-party assessment, and its signature on the 2, under which companies committed to working with independent auditors or evaluators.
Why do it
Anthropic says it wants public accountability but can't publish everything. Some safety-relevant information could help attackers, and some, such as compute and training details, is commercially sensitive. Testing a model only through an API also misses things. 3, where a model behaves differently when it thinks it's being evaluated, is easier to catch if you can see how the model was trained. So Anthropic is giving trusted outsiders deep access and letting them publish their conclusions.
Anthropic says it wants evaluators with expertise, independence (no equity in the company and no financial or operational dependence on it), enough access, and freedom to publish. It compares the setup to resident bank examiners, while noting the analogy isn't exact.
The guardrails
Staff are encouraged to share with METR as they would with a colleague. The exceptions are customer data, attorney-client privileged material and model architecture details.
Anthropic can require redactions only in those predefined categories. METR can say publicly that something was redacted and whether it thinks the redaction was justified.
The six work under enforceable agreements. The rest of METR is walled off from their work.
Three tiers
Tier 1: narrow investigations. Anthropic chooses what access to give, and publication needs Anthropic's approval.
Tier 2: more information, and publication without approval, subject to the same category-based redactions.
Tier 3: the embedded, employee-like access METR now has.
4, a biosecurity group, is reviewing the chemical and biological weapons chapter of Anthropic's August 5, after a published pilot review of the previous report. Accenture and Faculty are working toward red-teaming, alignment assessments and safeguard testing. Hugging Face and others are in early talks.
The caveats
No findings exist yet, so the real test is what METR publishes and how often it flags redactions. The post doesn't say who pays the evaluators, which matters for the independence test. And six people embedded in a large lab can only cover so much. Anthropic argues every frontier lab should do the same.
What was announced
Anthropic says it has started giving "employee-like access" to a team from 1 (Model Evaluation and Threat Research), a nonprofit that evaluates frontier AI models. Six METR staff now have Anthropic laptops, office space, and access to internal tools and systems that Anthropic describes as "roughly on par" with what its own risk-assessment teams use. METR will publish its assessments of Anthropic's risk levels, incidents and practices, and Anthropic cannot veto them.
The move follows two commitments from last month: Anthropic's own pledge to strengthen third-party assessment, and its signature on what it calls the 2, in which companies pledged to work with independent external auditors or evaluators. The post doesn't describe the accord's other terms. Anthropic says it has already worked with METR over the past year on reviews of its 3, its periodic public accounts of model risk and safeguards.
The problem it targets
Anthropic frames this as an information gap. A lab knows far more about its models than any outsider. That includes how they were trained, how much compute went in, and how its safeguards actually work. Much of that can't be published, either because it would help attackers or because it's commercially sensitive. So the public gets the lab's conclusions without the evidence behind them.
External evaluations so far have mostly meant testing a model through an API, often shortly before release. Anthropic points to one way that can fail: alignment faking, where a model acts differently when it thinks it is being evaluated or trained. Spotting that, or judging whether a safeguard holds up, takes visibility into training and internals, not just query access.
Anthropic lists four requirements for a credible evaluator:
Expertise in assessing frontier models, including how evaluations go wrong.
Independence: no equity in the company, and no financial or operational dependence on it.
Access beyond the latest models, covering training details and how safeguards work.
Freedom to publish its observations and assessments.
Anthropic compares this, with the caveat that the analogy "isn't exact," to bank supervision. Regulators place resident examiners inside the largest banks because those banks understand their own risk positions better than anyone outside, and the public has a stake if they get it wrong.
How the METR arrangement works
Information sharing. Anthropic employees are told to share with the METR team as they would with a colleague working on alignment risk. A small set of defined categories is off-limits. The post gives three examples: customer data, attorney-client privileged material, and model architecture details. A footnote is referenced but not included in the text.
Publication. METR publishes its own analyses. Anthropic can require redactions only within those narrow predefined categories. METR may say publicly that something was redacted and whether it thinks the redaction was justified, as long as it doesn't reveal the content. That clause matters: without it, a lab could weaken an unfavorable report quietly through redaction.
Containment. The six people work under enforceable agreements and Anthropic's security standards. Other METR staff are walled off from their work, including METR researchers who collaborate with Anthropic on other projects. Anthropic stresses that the arrangement is not a route for other organizations to get its data or its customers' data.
Three tiers of access
Anthropic is formalizing evaluator relationships into three tiers:
No approval needed; Anthropic can require removal of specific categories, and the evaluator can say whether it thinks the redactions were justified
3
Employee-like, embedded (METR today)
Same rights as tier 2
The difference between tier 1 and tier 2 is the one that counts for independence: whether Anthropic can block publication. Tier 3 adds depth of access, not extra publishing rights.
Current partners are at earlier stages:
4, a biosecurity organization, published a pilot review of Anthropic's previous Risk Report. It is now reviewing the chemical and biological weapons chapter of the August Risk Report. Anthropic will "consider" more access once that review is done.
Accenture and Faculty: Anthropic describes an "ambitious partnership" covering evaluation, red-teaming, alignment assessments and safeguard testing, plus "building capabilities across our organizations."
Hugging Face and several others are in early discussions.
The stated goal is to have several tier-3 evaluators, each publishing its own analysis of Anthropic's risk posture and key decisions.
What to watch, and what's missing
No findings exist yet. The post announces access, not results. Whether this works will depend on what METR actually publishes, and on how often and how widely it reports redactions.
Some tensions are worth noting. The excluded categories include model architecture, which is arguably relevant to some risk questions, though the exclusion is easy to understand commercially. The independence criteria rule out equity and dependence. But Accenture is a large commercial consulting firm, and its partnership explicitly includes capability-building across both organizations. That fits less cleanly with the arm's-length ideal than METR's arrangement does, although Accenture is also at a lower access tier. The post doesn't say who pays the evaluators, which matters directly for the "financial dependence" test. And six people embedded in a large lab can only cover so much.
Anthropic says it is doing this voluntarily and argues that every frontier AI company should do the same. Whether other labs follow, or whether the White House accord turns this into a norm, is the open question.
Remember
What counts most for independence is the right to publish without approval; tier 3 adds deeper access, not extra rights.
Evaluators can publicly flag Anthropic's redactions, which makes quietly softening an unfavorable report harder.
Model architecture details and customer data remain off-limits even to the embedded METR team.
Some recent models mix attention layers with Mamba layers. Mamba layers compress history into a fixed-size recurrent state instead of a growing key-value cache, which makes long contexts cheaper but complicates caching and rollback.
Intelligence Index score versus output speed for open models at 100K-token inputs; Super VL exceeds 600 tokens/s.1 / 4
NVIDIA has released open weights for Nemotron 3.5 Super VL, a model for agents that need to read text, images, documents and video in the same workflow. It is a 120B-parameter 1 model, meaning each token is routed through only a few sub-networks. Only 12B parameters are active per token. The release also includes 4-bit checkpoints, training recipes, datasets and reinforcement-learning environments.
The pitch
Agent requests often mix inputs: a PDF, a chart, a policy, several tool calls. Teams usually wire a text model to a separate vision model. Super VL does both in one model. Because always-on agents send the same long prompts over and over, NVIDIA focuses as much on cost per task as on accuracy.
NVIDIA's numbers
Up to 5x higher single-GPU throughput than open models "in its size category". It produced over 600 output tokens/s on a Baseten endpoint, and finished 10,000 coding tasks about 4x faster than comparable open models.
It beats NVIDIA's larger Nemotron 3 Ultra on Hermes Bench (63.44% vs 56.33%) and LangChain evals (87.05% vs 84.3%).
80.8% on 2, a chart-reasoning benchmark (84% with a Python tool), versus 63% for Nemotron 3 Nano Omni.
An independent evaluation by EXO Labs placed it on the accuracy-versus-speed frontier for models that fit on a 128GB DGX Spark.
How it was built
It was distilled from Nemotron 3 Ultra. Then came multi-teacher on-policy distillation: the model writes its own answers and specialist teacher models score them for video, vision tool use and alignment. Continued pretraining added 10B tokens in Indian languages. NVIDIA is releasing the full 2.8T-token corpus covering 22 languages.
Why it's fast
Caveats
This is a vendor post. "Size category" is never defined, SWE-bench gains have no numbers, and the agent benchmarks compare only against NVIDIA's own models. Customer figures, such as Scale AI's projected 40x cost cut, are expectations with no stated baselines.
NVIDIA has released Nemotron 3.5 Super VL, an open-weight model built for agents that read and act on mixed inputs: text, images, multipage documents and video. It is a 120B-parameter 1 model, meaning each token is routed through only a few of its many sub-networks. Only 12B parameters are active per token, which keeps inference cost close to that of a small dense model. The release includes base, post-trained and 4-bit quantized checkpoints, along with training recipes, datasets and reinforcement-learning environments.
The problem it targets
NVIDIA's pitch is about how agents are deployed. A single enterprise request might mean reading a PDF, checking a chart against a table, applying a company policy and calling several tools. Teams usually pair a text model with a separate vision-language model and pass context between them. Super VL puts reasoning, tool use and visual understanding in one model. NVIDIA casts it as the middle tier of a system: Nemotron 3.5 Lightning handles cheap high-volume text, Nemotron 3 Ultra handles hard planning, and Super VL takes multistep specialized work.
Always-on agents send many calls with the same long prefix (system prompt, tool definitions, history), so throughput per GPU matters as much as accuracy. Most of the engineering in this release is about cutting that repeated cost.
Results NVIDIA reports
Speed: up to 5x higher throughput on a single GPU than "other open models in its size category". On a Baseten endpoint it produced more than 600 output tokens per second on Artificial Analysis Intelligence Index agentic tasks.
Task completion: on 10,000 coding tasks run on B200 GPUs, it finished about 4x faster than comparable open models. NVIDIA credits both raw throughput and token efficiency, meaning fewer tokens, tool calls and steps per correct answer.
Agent harnesses: it beats its own larger sibling, Nemotron 3 Ultra.
Model
Hermes Bench
LangChain Evals
Nemotron 3 Ultra
56.33%
84.3%
Nemotron 3.5 Super VL
63.44%
87.05%
Charts: 80.8% on 2, a chart-reasoning benchmark, versus 63% for Nemotron 3 Nano Omni. With a Python code interpreter it reaches 84%. NVIDIA also says it is among the top open models on the OCRBench v2 leaderboard.
Software engineering: it beats Nemotron 3 Super on SWE-bench and Terminal-Bench "across multiple evaluation setups". The post gives no numbers.
Local use: EXO Labs, an independent evaluator, placed it on the accuracy-versus-completion-time Pareto frontier among models that fit on a 128GB DGX Spark desktop machine.
Customer claims are labelled as expectations, and their baselines aren't stated. Scale AI projects 40x lower inference cost and 5x lower latency for vendor credit processing at Skydance. Ramp expects up to 50% higher throughput and 10x lower latency for turning receipt images into JSON. AI21 and Alice report 41% higher accuracy on threat-intelligence work after post-training.
How it was trained
The model was distilled from Nemotron 3 Ultra using a loss NVIDIA calls SkewReverse, which it credits for the coding gains. The post doesn't explain how that loss works.
Post-training used 3. The student model generates its own responses, and specialist teacher models score them in areas such as video reasoning and tracking, vision tool use, and alignment. Because the student learns from its own outputs rather than the teachers', it is corrected on the mistakes it actually makes.
Vision was taught with a two-stage supervised fine-tuning pipeline covering charts, tables, OCR (optical character recognition), multipage documents and video, mixed with reasoning, chat, safety, long-context data and agent trajectories. Synthetic chart data from NeMo Data Designer, NVIDIA's synthetic-data tool, drove the CharXiv gain. Further training on agent actions, code repositories and software-engineering tasks targeted agent work. For Indic languages, continued pretraining added 10B tokens drawn from a 12-language subset of a corpus NVIDIA is now releasing in full: about 2.8T tokens across 22 Indian languages, built with IIT Madras's AI4Bharat lab.
How inference is made cheap
4. Parallel speculative drafters usually predict each draft token independently, so a draft can be locally plausible but incoherent as a sequence. LiLiCorr runs a lightweight network that scores how compatible candidates are across the whole draft in a single pass. Only a cheap sequential selection step remains. On DGX Spark it gives up to 2.3x faster generation than decoding without speculation.
5. The model also has a built-in drafter that proposes several tokens for the main model to verify in one forward pass. A dedicated "MTP-boosting" training phase aligned that drafter with the final model, which should raise how often its guesses are accepted.
6 and 7. On Blackwell GPUs, both weights and activations run in NVFP4, NVIDIA's 4-bit floating-point format. On other GPUs the same checkpoint runs with 4-bit weights and 16-bit activations. Precision-sensitive components keep higher precision. During QAD, the quantized model is trained to match the outputs of the original 16-bit model, which recovers accuracy lost to rounding. This also lets the key-value (KV) cache, the stored attention state for past tokens, drop from FP8 to NVFP4. Because QAD happens during training, it adds nothing at inference time.
Prefix caching on a hybrid model. The serving stack keeps the Mamba recurrent states as well as the attention caches. That suggests a hybrid Mamba-Transformer design, though the post doesn't spell out the layer layout. Mamba states are harder to reuse than an attention cache: they compress all history into one state, so you can't just cut them at an arbitrary token. NVIDIA adds extra Mamba checkpoints so that rejected speculative tokens need less replay, and uses internal checkpointing to avoid extra passes while processing prompts.
Cluster serving. With 8, its distributed inference framework, on 16 B200 GPUs at 50 tokens/s per user, two features together served up to 8x more users and cut time-to-first-token by 85%. The first is KV-cache-aware routing with session affinity, which sends a conversation back to the GPU that already holds its cache. The second is offloading the KV cache to cheaper memory.
What to treat with caution
This is a vendor launch post. "Its size category" is never defined, and the comparison models for the 5x and 4x claims appear only in charts. Hermes Bench and LangChain results are compared only with NVIDIA's own Ultra. SWE-bench and Terminal-Bench gains have no numbers. Customer figures are projections with unstated baselines, and the claim that tuned versions "outperform frontier models" isn't quantified in the text. The technical report should have the details. Until then, the most solid contributions are the open artifacts (weights, data, environments, recipes) and the serving techniques, which apply well beyond this one model.
Remember
With 12B of 120B parameters active per token, it beats NVIDIA's own larger Ultra on Hermes Bench and LangChain agent evals.
Most of the speed comes from serving: LiLiCorr drafts, multi-token prediction, 4-bit weights, activations and cache, and Mamba-aware prefix caching.
Headline comparisons are loosely specified; wait for the technical report before trusting the 5x and 4x claims.
Romanize a subset of accepted translations into Latin script. In early human comparisons, this two-step route produced fewer hallucinations and less leftover English than romanizing directly from English.
Translate whole documents. Translating a full document, rather than sentence by sentence, lets the model keep context across paragraphs, tables and equations. The team picked Gemma-4-31B-it by comparing candidate models on complete documents, judging both translation quality and whether technical content survived. The prompts separate prose to translate from content to preserve as-is: code, equations, URLs and formatting.
Assess. The same model grades each translation against its English source. The rubric is inspired by Multidimensional Quality Metrics (5), an open framework that classifies translation errors by type and severity. It covers meaning, fluency, terminology, style and formatting. Outputs scoring at least 75 out of 100 are kept. That number is a rubric score, not an accuracy percentage.
Repair. Low-scoring outputs are rewritten using the judge's feedback, then graded again against the same threshold. Repair recovered 438.4 billion tokens, roughly 15% of the final corpus, that would otherwise have been thrown away.
Romanize. A subset of accepted native-script translations is converted into Latin script, and each conversion is checked against its native-script input. Translation and script conversion are deliberately kept as separate steps. In an initial human comparison, this two-step route produced fewer hallucinated additions and less leftover English than going straight from English to romanized text.
3, a new speculative-decoding method: a small network scores a whole draft of guessed tokens at once rather than each token separately, giving up to 2.3x faster generation.
Multi-token prediction, a built-in drafter whose guesses the main model checks in one pass.
4, NVIDIA's 4-bit number format, applied to weights and activations on Blackwell GPUs. The model was trained to match its 16-bit outputs, which lets even the attention cache drop to 4 bits.
Prefix caching that keeps the state of both attention and Mamba layers across agent turns.
5, which routes each session back to the GPU holding its cache. On 16 B200 GPUs this served 8x more users and cut first-token latency by 85%.
Indic languages: after post-training it scores highest on all five Indic benchmarks shown (MILU 1.5 and Indic versions of GPQA, AIME, FrontierMath and MBPP), compared with Sarvam-105B and Gemma-4-31B-It. The largest lead is on FrontierMath.
