Read the Traces, Distrust the Blobs · Morning Gist
Friday, October 25 stories · 11 min
Read the Traces, Distrust the Blobs
Today's theme is trust boundaries: eval tools that skip the data, agent code treated as untrusted, replayable reasoning blobs, and federated learning that now relies on enclaves.
Picked from 39 new articles across 24 sources · arrow keys for other days
Today in 30 seconds
30s left
Hamel Husain finds Anthropic's claude-api eval plugin good at spotting issues, but it picks evaluators before anyone has read the traces. story 1 ↓
Raschka benchmarks TypeSafe AI's Jev, a classifier that needs no fine-tuning: 96.5% on IMDb for $0.65, compared against fine-tuned baselines. story 2 ↓
NVIDIA's TensorRT Model Connect uses coding agents across 128 model families, treating their output as untrusted. Validation becomes the bottleneck. story 3 ↓
OpenAI says Moonshot-linked accounts replayed encrypted reasoning blobs to extract hidden chain-of-thought. The campaign peaked at 16,000 requests before being disrupted July 28. story 4 ↓
Gboard's federated learning now uploads encrypted raw examples to attested TEEs. Only DP weights leave, so privacy depends on the enclaves. story 5 ↓
Simon Willison tests GPT-6.1 Sol, which OpenAI pitches as near-Astra at coding and computer use for one-fifth of Astra's token price.
1/5Hamel Husain· Hamel Husain· Sep 30
Hamel Husain: Claude's New Eval Plugin Writes Evals Before You Read Data
In a livestream on leasing-assistant traces, Husain found the claude-api plugin good at spotting issues, but it chose evaluators before anyone had read the data.
Why it matters · Anthropic's own eval tooling will shape how many teams start writing evals. Husain's walkthrough also works as a checklist for any auto-eval tool: does it get you to read the data first, keep each grader to one failure, and show you the judge prompt?
Claude recommends a call-transfer evaluator before the user has looked at any conversations.1 / 3
Anthropic's claude-api plugin for Claude Code now includes build_eval and hill-climb commands. They propose evals, check the graders, and then iterate your application against them. Hamel Husain and Isaac Flath tried the plugin on a livestream, using conversation traces from an apartment-leasing assistant. Husain says a first-party tool deserves a review because it will shape how people approach evals.
Where it went wrong
Evals came before error analysis. Claude listed candidate failures right away and asked the pair to pick one to build an eval for. It recommended call-transfer rules. Neither of them had read the conversations yet, so they couldn't tell whether this was a real failure or a priority. They took the default anyway, as a typical user would. Husain argues that error analysis on the data should decide which failures get evals at all.
Label review happened without context. To check labels, Claude dumped conversations into inputs.md and asked the pair to skim the file and report wrong labels in chat. That meant reading long transcripts in an editor and sending corrections somewhere else. Husain calls it "the most painful way you can think of to label data." The coding agent could have built an annotation web app, and they ended up asking Claude to build one. Later, Claude showed only aggregate label counts and asked, "Would you have scored any case differently?" That question can't be answered without seeing the cases. Husain says this was a recurring pattern: the tool rushed to create artifacts and ask for approval before helping the user understand the data.
2/5Ahead of AI (Sebastian Raschka)· Sebastian Raschka, PhD· Sep 29
Raschka tests Jev, a general-purpose classifier: 96.5% on IMDb for $0.65
Raschka walks through text classifiers from bag-of-words to TypeSafe AI's Jev, guesses how it is built, and runs IMDb benchmarks against fine-tuned baselines.
Why it matters · Jev claims GPT-class decision quality at classifier cost and latency, with no fine-tuning. If that holds, routing, triage, injection screening and LLM-as-judge steps in agent harnesses get much cheaper, and you need a stronger reason before training a custom classifier.
Overview of the Jev API: typed questions and criteria go in, labels with probabilities come out.1 / 4
Jev is a proprietary model from TypeSafe AI, a startup that just came out of stealth. It answers typed questions about any input through three APIs: Choice (multi-class), Noul (a yes-probability per question, so it handles multi-label) and Score (ordinal rubric). TypeSafe claims Jev matches GPT-5.6 Luna on decision tasks while being orders of magnitude faster and cheaper. Raschka started out thinking it was "just a classifier" and ended up impressed. He has no affiliation with the company.
3/5NVIDIA Technical Blog· Tanya Lenz· Sep 29
NVIDIA's lessons from building a TensorRT project around coding agents
TensorRT Model Connect covers 128 model families with little orchestration. Its architecture is built around parallel work, isolated model families, and validation that treats agent output as untrusted.
Why it matters · If you're adding agents to a codebase, this is a concrete example of the architectural choices that decide whether cheap candidate code turns into reliable software. It also names where the approach breaks down: validation becomes the bottleneck.
The pipeline: human intent and a high-level goal feed N agent runs, which pass validation before code counts as verified.1 / 2
TensorRT Model Connect is NVIDIA's open source collection of C++ reference implementations on top of TensorRT. It turns Hugging Face or local checkpoints into versioned .bundle artifacts and exposes native C++ APIs organized by task: text, vision, audio, diffusion, embedding, forecasting and others. As of the July 29, 2026 release it covered 128 model families tested on GB300. The team says plainly that this number doesn't measure agent productivity. The more useful part of the post is the engineering pattern behind it.
Choose problems that split into parallel work
The first lesson is about which problems to pick. Supporting the long tail of models splits naturally into independent pieces: model families, configs, operators and validation cases rarely block each other. The team argues that when a task has a long serial critical path, adding agents mostly adds coordination overhead and lets errors cascade.
Agent runs start from an outcome plus the evidence needed to accept it, not a step-by-step recipe. Examples are "support this family" or "close this accuracy gap," checked against an established reference implementation and project tests. The setup is deliberately minimal: a general-purpose coding agent, repository instructions and strict validation gates. Extra constraints are added only after repeated failures show they're needed. In the authors' words, "The implementation path is flexible. The acceptance criteria are not."
A campaign starting July 1 used cross-conversation replay of encrypted reasoning blobs. It peaked at 16,000 requests from 4,000+ users and was disrupted by July 28.
Why it matters · If your system hands clients encrypted or opaque reasoning artifacts to pass back later, treat those artifacts as an attack surface. A model that can read them can be asked to transcribe them.
Image: OpenAI
OpenAI reports that it found and shut down a coordinated adversarial distillation campaign aimed at its models' protected reasoning, meaning the internal working-through of a task that is withheld from the final answer. It attributes a core cluster of the activity to individuals associated with Moonshot AI, the developer of Kimi. OpenAI says it is unclear whether every operator it observed belonged to a single actor.
The mechanism
Nothing was hacked in the conventional sense. OpenAI says no encryption was broken, no database was compromised and no stored conversations were accessed. The attackers got the model itself to reproduce its hidden reasoning in a visible form.
The clearest technique described is a replay attack. An operator copied the encrypted reasoning blob from one conversation, pasted it into another, and asked the model to decrypt and transcribe it. The model can legitimately consume that artifact, so it could be talked into revealing what was inside. Independent security researchers separately reported related cross-model and conversation-compaction vulnerabilities through responsible disclosure. OpenAI confirmed those attack paths were real. The post gives no technical detail on the compaction variant.
Scale and timeline
July 1: activity begins at low volume.
July 24–25: spikes of 16,000 requests using an extraction pattern, from more than 4,000 users.
5/5Google Research· Oct 2
Google moves federated learning onto attested TEEs, uploading raw encrypted examples
Gboard's next-word models now train server-side inside TEEs. Access policies are public, binaries build reproducibly, and only DP weights leave the enclave.
Why it matters · This changes what federated learning means: devices now upload encrypted examples instead of computing gradients locally, so privacy rests on attested TEEs and DP rather than on data staying on-device. That moves the compute ceiling and the threat model.
Image: Google Research
Google's next-generation federated learning (FL) system runs server-side training inside Trusted Execution Environments (TEEs). The aim is to make the anonymization guarantee checkable by outsiders, not just something users take on trust. Gboard already uses it for English and Japanese next-word prediction.
How it works
The system has four parts:
Upload under a policy. Devices encrypt their training examples locally and upload them. Each upload carries a pre-authorized access policy: the set of TEE computations allowed to touch that data, all of which release only anonymized results. Policies must be published to Rekor, a public transparency log, so auditors can see every workload a device could take part in.
Key release. A Key Management System, itself a cluster of TEEs running RAFT, hands decryption keys only to attested workloads that match the policy. Data can be decrypted only for a limited time after upload.
Execution. A "root" TEE runs a Python training loop and hands parallel subtasks to worker TEEs, coordinated with Federated Language, an open-source descendant of TensorFlow Federated. The only outputs are metrics and differentially private model weights.
Recovery. Each round ends with a KMS-encrypted checkpoint, so the job can survive failures without leaking anything new.
The KMS and data-processing binaries build reproducibly from the open-source Confidential Federated Compute repo. To keep model architectures and preprocessing proprietary, the TEEs can serialized logic at runtime. The privacy guarantee holds only if all privacy-relevant code, such as the DP noise addition, stays hardcoded in the published Python program. Google concedes that side channels remain an open problem for these dynamically loaded workloads.
One grader covered four failures. The resulting protocol_ok metric passed a case only if all four of these checks passed:
Confirmation was asked exactly once (LLM-as-judge)
Nothing was said between the caller's consent and the transfer (code)
Nothing was said during or after the transfer (code)
Tool mechanics, such as "triggering" a transfer, were never said aloud (code)
Husain would scope each eval to one error, or at least split the code-based checks from the judge-based ones. He also found Claude's prose description of the metric hard to follow. He wanted to see the code and the judge prompt themselves, which he says are always worth reading.
What worked
Husain says the plugin's one-shot issue discovery was the strongest he has seen. It found problems with human handoff, formatting and voice-agent behavior that other auto-eval approaches had missed. He also agrees with the principles in Anthropic's launch post: look at the data, sample intelligently, and don't saturate your own evals.
His verdict for now is to hold off. He prefers the less opinionated eval skills he built with Shreya Shankar. The plugin's author has said he will act on the feedback, so the tool may change soon.
Remember
The plugin's one-shot issue discovery was the best Husain has seen, catching handoff, formatting and voice-agent failures.
Its workflow picks an eval and asks for label approval before you've read traces. Do error analysis first.
Keep each grader to one failure, separate code checks from LLM judges, and read the judge prompt.
He first rebuilds the history of text classification on the 25k-review IMDb test set:
Bag-of-words + logistic regression: 89.9%
LSTM trained from scratch: 85.66% (overfit)
Text CNN: 90.07%
GPT-2 124M with a classification head: ~92%
ULMFiT (RNN, pretrain then fine-tune): 95.4%
ModernBERT fine-tuned with little tuning: ~95%
Jev's Choice API scored 96.47% in 22 minutes, using 15.5M input tokens at a total cost of $0.65. The Noul API scored 96.20%. Repeated runs vary slightly because serving is non-deterministic. ModernBERT needed 23 minutes to fine-tune and 7 minutes to evaluate on a DGX Spark. Raschka cautions that IMDb may have been in Jev's training data. Clones did worse: Contrastive Language Models scored 82.90% and Laya 92.33%, and both failed his real-time Tetris test, which Jev passes. One reader reports Jev is weak on legal reviews in languages other than English.
How it might work
TypeSafe has not disclosed the architecture. Raschka guesses a small ModernBERT-like encoder, which would explain the low latency. To show how a Jev-style API can be added to a BERT or GPT backbone, he swaps the output layer for a one-node scoring head. Each (text, instructions, candidate description) triple produces one scalar score. A softmax across candidates gives the class probabilities, so the number of labels can change without changing the architecture. Building this is easy. Raschka argues the hard part is making it generalize, and the CEO's statement that "100% of our data is synthetic" points to data curation as the real work.
TypeSafe says it trained Jev with "RL for Calibrated Decisions" (RLCD), which is unpublished. Raschka compares it to RLCR (2025). RLCR replaces the binary RLVR reward with correctness minus a Brier penalty, c − (q − c)². On HotpotQA, RLCR cut expected calibration error (ECE) from 0.37 to 0.03, with accuracy at 62.1% versus 63.0%. When Raschka added a Brier term to ModernBERT's cross-entropy loss, calibration improved only a little over temperature scaling, and ECE got worse on one of nine datasets. He notes that cross-entropy already rewards calibration in theory. The Brier term matters more in RL, where no cross-entropy loss is being minimized.
OpenAI has since announced a similar Decision API built on Luna.
Remember
Jev scores 96.47% on IMDb for $0.65, above a fine-tuned ModernBERT (~95%), though the test set may be contaminated.
A Jev-like API is a one-node scoring head plus a softmax over candidates. Generalizing across tasks takes heavy data curation.
Calibration rewards (RLCR) cut HotpotQA calibration error from 0.37 to 0.03. A Brier term added to cross-entropy helped little.
The architecture separates layers that change at different speeds. TensorRT and CUDA are the stable foundation. Model Connect is the fast-moving integration layer. Each model family owns its own builders, runtime pipelines, helper kernels, config and validation evidence. Code moves into shared infrastructure only when several independent owners need the same "assumption-free contract." The team accepts some duplication between similar families as the price of fewer merge conflicts and contained failures. Changes are kept reversible, which the post calls "two-way doors."
Validation is the real constraint. Semantic interfaces (text in, image out) make final behavior easy enough for a human to spot-check. If automated checks pass but a person finds a bad artifact, the team treats it as a false success. They reproduce it, encode the missing invariant, and harden the standard operating procedure. QA works from an organizationally independent position on the same CI pipeline and acts as a red team trying to falsify claims.
The team lists what remains unsolved. Not every task splits into independent units. Shared build and CI infrastructure can still break many families at once. Reference implementations are not infallible oracles. Most tasks are still started by humans. And more parallel agents can increase validation demand faster than they increase accepted throughput. The project is in public preview.
Remember
Agent leverage depends on choosing problems that split into parallel work; serial critical paths mostly add coordination overhead.
Give agents outcomes, references and fixed acceptance gates; add orchestration only when repeated failures justify it.
Candidate code scales with tokens, but trusted code scales only as fast as validation. Isolation limits how far failures spread.
Further investigation linked a cluster of more than 15,000 users through shared prompt patterns. That cluster was fully disrupted by July 28.
OpenAI notes that the technique changed over the course of the campaign.
Fixes
The account-level measures were bans, tighter signup and infrastructure controls, and wider network monitoring. OpenAI also lists these technical changes:
It closed the pathway that let someone holding another user's encrypted reasoning replay it and recover the contents.
It hardened reasoning isolation across users, workspaces, organizations and model families.
It added checks that detect and hold streamed output that might expose reasoning.
Where activity ran through third-party services, OpenAI worked with those providers. It also shared findings through the Frontier Model Forum and government channels.
OpenAI argues that the risk is mainly about safety. A model trained on extracted reasoning would not inherit the original model's output safeguards, and distillation lets capabilities spread without matching safety investment. It says the problem applies broadly: "systems that support portable or replayable reasoning artifacts may face related risks."
OpenAI lists remaining gaps. Partner-hosted deployments need the same protections as first-party ones. Tool-output attacks need defenses that inspect more than ordinary visible text. Work also continues on classifiers and model refusals.
The post does not quantify how much reasoning was actually extracted, and it gives no evidence for the attribution.
Remember
Attackers replayed encrypted reasoning blobs in fresh conversations and asked the model to transcribe them; no cryptography was broken.
Peak: 16,000 requests from 4,000+ users on July 24–25; a linked 15,000-user cluster was disrupted by July 28.
OpenAI attributes a core cluster to Moonshot AI but gives no evidence and no estimate of how much leaked.
Earlier, server operators had to be trusted to add DP noise correctly. Secure Aggregation hid individual updates but was incompatible with state-of-the-art central DP. The new design keeps central DP and makes it verifiable.
The bigger change is that client gradient computation moves to the server. Because all uploads are collected before training starts, daily swings in device availability stop mattering. The training program can work out the best device participation schedule itself and tune DP parameters around it. Training that used to take 1–2 months per model, held back by device availability, on-device compute, and competing workloads, is now limited mainly by how many TEEs are available. Google reports "substantially faster" training plus better accuracy and stronger privacy, but gives no accuracy figures or DP epsilons.
Next steps include TEEs paired with accelerators for larger models, synthetic data generation, and combining these TEEs with ones built for LLM inference. Google says it eventually hopes to provide formal correctness proofs for the DP implementations.
Remember
Devices now upload encrypted examples and gradients are computed in server TEEs, which removes on-device compute and availability limits.
Public Rekor-logged access policies, reproducible builds, and attestation-gated keys let outsiders verify that only DP weights get out.
Per-model training used to take 1–2 months. Google reports large speedups but publishes no accuracy or epsilon numbers.
