Small models, tight harnesses and unreviewed scans: today's stories are about what AI work really costs and how far raw model output can be trusted.
Picked from 20 new articles across 24 sources · arrow keys for other days
Today in 30 seconds
30s left
Claude Haiku 5.5 matches GPT-6 Luna's price and beats it on most evals, but uses about 3x the tokens. story 1 ↓
NVIDIA took second at KDD Cup on a fixed small LLM by narrowing tools, preprocessing inputs and reading execution traces. story 2 ↓
Anthropic is scanning open-source projects for free with unreviewed Claude output. Experts found one false positive in 97 serious findings. story 3 ↓
Only 1.1% of 857 releases from nine Chinese labs shipped safety results, SemiAnalysis finds. Beijing regulates only applications. story 4 ↓
A Hugging Face author trained six models for about $103 with the ML Intern agent. Writing the prompt took most effort. story 5 ↓
LegalOn cut its estimated daily Codex costs 65% by routing each task to OpenAI's Astra, Sol or Luna model.
1/5Latent Space· Oct 8
Claude Haiku 5.5 matches GPT-6 Luna's price and beats it on most evals
Anthropic's first small-model update in a year scores 43 on Artificial Analysis's index. It also burns about 3x Luna's tokens, and prompts over 100K tokens cost 5x more.
Why it matters · The cheap subagent is often the model that sets the bill in an agent harness. Haiku 5.5's independent numbers show the sticker price and the real cost per task can differ a lot.
Artificial Analysis Intelligence Index: Haiku 5.5 scores 43, up 26 points from Haiku 4.5, above GPT-6 Luna's 38.
Anthropic released Claude Haiku 5.5, its first Haiku update since Haiku 4.5 on October 15, 2025. Its price matches OpenAI's GPT-6 Luna exactly. Anthropic pitches it as the cheap worker inside multi-model agents: Opus or Sonnet 5.5 plans, and Haiku handles summaries, context compaction and database queries.
Tiered pricing: under 100K prompt tokens it costs $0.10 input / $0.50 output per 1M tokens, and cache reads cost $0.01. Above 100K, every price goes up 5x ($0.50 / $2.50, cache reads $0.05). Anthropic says it costs about 75% less to run than Haiku 4.5 on average. Context grew from 200K to 1M tokens, with roughly 12K max output tokens. It is the first Haiku with effort settings and adaptive thinking.
Independent numbers
Artificial Analysis published the most detailed third-party evaluation:
Intelligence Index: 43 at max effort, up 26 points from the last Haiku. That puts it ahead of GLM-5.3 Flash (42), Gemini 3.8 Flash (41) and GPT-6 Luna (38), close to the 2.8T-parameter open-weights Kimi K3 (44), and 13 points behind Sonnet 5.5 (56).
Terminal-Bench 4.0: 33%, up from 0%. Luna scores 13% and Gemini 3.8 Flash 20%. Anthropic's own figure is 39%. Anthropic also reports OSWorld rising from 15% to 72%.
AA-Omniscience: 36% accuracy with a 40% hallucination rate. Luna gets 44% accuracy but hallucinates 77% of the time, and Gemini 3.8 Flash is 55% / 55%. Haiku answers fewer questions correctly partly because it says "I don't know" more often.
35%, versus 53–60% for its peers. A pre-release safety bug makes the model refuse too often. Anthropic is fixing it, and the eval will be re-run.
2/5NVIDIA Technical Blog· Jiwei Liu· Oct 8
NVIDIA's KDD Cup runner-up: a small LLM, a tightly constrained harness
With the model fixed by the rules, NVIDIA's KGMON team took second place mostly by narrowing tools, preprocessing inputs and reading execution traces.
Why it matters · Most production agents also run on models you can't retrain. This is a practical breakdown of where reliability came from once the harness was the only thing the team could change.
Overall harness: preprocess, constrain, persist, with traces feeding evaluation back into the design.1 / 4
NVIDIA's KGMON team placed second in the KDD Cup 2026 Data Agents competition. Agents had to answer natural-language questions over mixed sources: SQLite databases, CSV and JSON files, prose documents, PDFs and briefing videos. The rules fixed a small LLM for every team and allowed no internet access, so the harness was the only thing teams could really tune. Jiwei Liu's write-up argues that reliability came from narrowing what the agent could do, not from giving it more freedom. The post doesn't publish leaderboard scores or ablations, so the evidence for each choice is qualitative.
How the harness works
One query surface. CSV and JSON files were loaded as tables into the task's SQLite database. The agent got two custom functions, schema() and sql(query), so there was only one path to structured data.
Schema scouting. Before the reasoning loop starts, a read-only pass inspects columns, likely join keys, look-alike and duplicate fields, units, null patterns and row grain. The agent receives this brief up front, which saves a discovery turn and cuts errors from wrong columns, missed joins, or confusion about what each answer row represents.
3/5Anthropic Research· Oct 8
Anthropic offers open-source projects unreviewed Claude vulnerability scans for free
OSS Scanner skips human triage. In spot checks, experts found one false positive in 97 critical and high-severity findings.
Why it matters · Model-found vulnerabilities now pile up faster than people can verify them. Anthropic is betting that sending raw, unreviewed model output straight to maintainers does more good than harm, and the article includes evidence on how accurate that output is.
Image: Anthropic Research
Anthropic is launching OSS Scanner, a free, opt-in service. It runs its strongest models, including Claude Mythos, over important open-source projects on a regular schedule and sends the findings to maintainers with no human review. Anthropic says it is modeled on Google's OSS-Fuzz, with language models taking the place of fuzzers.
Why skip human review
The problem is throughput. Over six months of scanning major projects, Anthropic found more than 29,000 candidate vulnerabilities. Its staff has manually reviewed and triaged only about 6,000. Maintainers kept asking for everything unverified, and Anthropic has already sent nearly 5,000 unvalidated reports at their request. Anthropic argues that exploits can now be written in minutes, so getting findings to maintainers fast matters more than full validation. Human-verified disclosure through its existing coordinated vulnerability disclosure (CVD) process will continue, especially for projects that can't triage reports themselves.
The capability behind this has grown quickly. On the CyberGym benchmark, Anthropic says LLMs went from finding under 20% of vulnerabilities at the start of last year to over 85% this year.
Each report includes:
a self-contained reproducer
an explanation of the bug, with a bisection to the commit that introduced it where possible
a candidate patch, when available
During several weeks of testing with dozens of projects, the pipeline produced hundreds of reports. Several of the bugs could be chained into unauthenticated remote code execution.
4/5SemiAnalysis· Mark Chen· Oct 8
Only 1.1% of 857 Chinese model releases shipped with safety results
SemiAnalysis went through every release from nine Chinese labs and coded 102 expert texts. Beijing writes about loss of control but regulates only applications.
Why it matters · US arguments for and against pacing the frontier both rest on assumptions about China. This census measures what Chinese labs actually evaluate and publish before they ship, and finds that most publish nothing.
China's safety rules compared with the EU and California: stricter on content, looser at the frontier.1 / 4
Dario Amodei's September essay called for labs to pace the frontier, and US arguments over it rely heavily on China. SemiAnalysis's Mark Chen set out to measure what China actually does. His conclusion is that Beijing writes about frontier risk but governs for speed.
What the rules cover
On paper, China has gone further than anyone outside Europe. AI Safety Governance Framework 3.0, released by TC260 on 14 September, adds a risk category called "behavior deviating from expectations." It covers models that take privileges without authorization, sandbag during evaluations, or disable shutdown scripts. But the framework's first principle puts innovation as "the first priority." China shelved its comprehensive AI Law in 2025.
5/5Hugging Face· Oct 8
Six custom models for about $103 using Hugging Face's ML Intern agent
A Hugging Face author used the agent to distill, fine-tune and train LoRAs end to end. The prompt, not the training, was where the effort went.
Why it matters · This is a concrete look at agent-run training at hobby-level budgets, including what a good task spec contains. It's a vendor post, though, and the evals are self-reported in model cards.
Viewpoint Orbit LoRA rotating an object to requested camera angles with Qwen-Image 2.1.1 / 2
Hugging Face's ML Intern is an agent mode in HuggingChat. It plans a training project, asks for a budget, runs a small test, then trains, evaluates and publishes a model on HF hardware. The author of this post, which is partly a product showcase, used it to build six public models in a few days. Total compute was about USD 103.
What got built
Pocket Rewriter. Qwen-Image 2.1 ships a 9B prompt rewriter that needs about 20 GB and spends thousands of tokens thinking before it writes. The agent generated 8,797 image requests, had the 9B teacher rewrite them in 2h37m on one A100 (~$6.50), filtered those down to 1,840 examples, and distilled 0.8B and 2B students in 12 and 18 minutes on an A10G. The 0.8B student returns valid output 99.7% of the time, uses about a quarter of the teacher's tokens, and runs on a CPU as an 812 MB GGUF file. Total: $16.
Citrus Doctor. Qwen3.5-2B fine-tuned on 3,017 images covering 21 citrus pests, diseases and deficiencies. Accuracy on 335 test photos went from 14.9% to 52.8% after two epochs. Cost: $1.90.
Viewpoint Orbit LoRA for Qwen-Image 2.1. The agent rendered 1,030 Google Scanned Objects from 24 angles, then trained on 1,844 before/after pairs across 23 camera instructions. That took 2,000 steps in about 90 minutes on one A100. The project needed 48 jobs, including failed ones the agent resubmitted itself. Cost: $16.
The main caveat is token use. At max effort Haiku uses about 162K output tokens per Index task, roughly 3x Luna's ~50K. Going from xhigh to max adds 2 points for about 1.8x the tokens. At equal scores it is still wordier: Haiku at high effort reaches 38 with ~55K tokens, while Luna reaches 38 at max with ~50K. Combined with the 5x price step, this probably explains why Cursor reports "10x cheaper" on short requests while Anthropic claims 75% cheaper on average. Independent cost-per-task figures are not out yet.
Partner and practitioner reports
Cognition says it scores 58.4% on FrontierCode 1.1, ahead of Sonnet 5 at about one-eighth the cost per task. GitHub Copilot says it matched Sonnet 5 on many coding tasks using fewer steps. On ParseBench document parsing it is roughly even with Luna: better on tables, worse on charts and bounding boxes, at about $1.2 per 1,000 pages.
Anthropic made other changes the same day:
Sonnet 5.5 cache reads were halved to $0.10 per 1M tokens, which Anthropic says makes agentic work about 20% cheaper.
Max and Team plans now include monthly API credits ($100 to $500).
The Python and TypeScript SDKs now run the computer-use and browser-use action loop themselves, sending actions to drivers such as Browserbase and E2B.
Remember
Artificial Analysis Intelligence Index 43 vs. Luna's 38, at identical $0.10/$0.50 per-1M pricing under 100K tokens.
Uses about 3x Luna's output tokens at max effort, and prompts over 100K tokens cost 5x more.
Hallucinates far less than Luna (40% vs 77%) but recalls fewer facts; an over-refusal bug drags its automation score down.
A small, opinionated tool set. Besides schema and SQL, the tools were prose_helper() and write_answer(df), which writes the final answer atomically. A persistent Python environment keeps variables between calls. Middleware repairs malformed tool calls so a single bad call doesn't kill an attempt. Shorter valid attempts leave budget for more runs.
Prose kept out of context. Direct open()/.read() of whole files was blocked. The agent could only take bounded previews or run regex searches. Relevant chunks then go to a separate LLM call with temperature 0 and reasoning off. In answer mode it extracts a rule or threshold. In table mode it turns repeated records into a SQL table.
Video preprocessed offline. Keyframes are extracted and aligned with transcript segments before the run. This tied spoken instructions to the right slide values, which often included distractors.
Evaluation and the overfitting trap
Every attempt logs prompts, tool calls, SQL, intermediate results, repairs and answers. An inspector agent sorts failures into categories: schema confusion, bad joins, missed prose rules, formatting and brittle prompt rules. To fit the exact value-level scoring, repeated attempts were grouped by answer values rather than column names, and contested tasks got extra runs. The author flags the token and latency cost of this.
Autonomous self-improvement loops sometimes hardcoded training examples into prompts, piled up contradictory instructions, or added brittle postprocessing. The team's fix was to require held-out tasks, prompt audits and human approval before any change was kept.
Remember
With the model fixed, the team got its gains from one SQL surface, a pre-run schema brief and four narrow tools.
Prose went through a separate temperature-0 extractor so raw document text never filled the main agent's context.
Self-improvement loops overfit the benchmark. Changes needed held-out tasks and human approval before being kept.
Anthropic's penetration testers checked 97 critical and high-severity findings from 48 projects. Of these, 85 (88%) met the bar for CVD. Of the other 12, 11 were real bugs that duplicated known issues or other findings from the scan. Only one was a false positive.
Maintainer reports point the same way. wolfSSL says 72 of the 74 reports it received were valid, and five became CVEs. OpenSSL's Anton Arapov says the raw model output was "as good and sometimes better" than reports from humans, and that reports with a working exploit are "basically job done." PostgreSQL's Noah Misch says some patches were usable nearly as-is, and early access let the team fix issues before they reached a general release.
The failures Anthropic admits to are inflated severity ratings and the scanner sometimes misreading a project's threat model. It makes no claims about recall, or about precision on lower-severity findings.
Enrollment is by PR to a GitHub repo. Eligibility follows OSS-Fuzz-style criteria: "critical impact on infrastructure and user security," decided case by case.
Remember
Anthropic has 29,000 candidate vulnerabilities but has triaged only about 6,000. OSS Scanner sends unreviewed model output directly to maintainers.
Spot check: 85 of 97 critical and high findings met the CVD bar, with one false positive. wolfSSL found 72 of 74 valid.
Known weaknesses are inflated severity ratings and misread threat models. The article gives no recall figures or lower-severity accuracy data.
Every binding Chinese rule since 2025 covers what AI outputs and how it is deployed: content labelling, agent rules, AI-companion rules, protections for minors. None is triggered by training compute or model capability. By contrast, EU duties start at 10²⁵ FLOP, and California's SB 53 requires frontier frameworks and 15-day incident reporting above 10²⁶ operations. In the five days after Framework 3.0, regulators issued seven texts, all at the application layer.
What the labs disclose
Chen built a dataset of 857 releases from nine developers, from 2021 to September 2026. A release counted as disclosed only if it came with a quantitative safety result tied to that specific model.
31 (3.6%) ever got a published safety result. Only 9 (1.1%) had one available at launch.
Results published after launch arrived a median of 42 days late. The longest delay was 349 days, for DeepSeek-R1.
94.9% have no safety disclosure of any kind.
Releases rose thirtyfold, to 101 in Q3 2025. Releases with any safety result never exceeded 7 in a quarter, and no policy milestone changed that.
Startups disclosed more often (6.3%) than the hyperscalers (2%). Tencent disclosed for 1 of 133 releases.
93% of reasoning models have no published results. No Chinese frontier text model has shipped with a full dangerous-capability evaluation. Zhipu's GLM-5.3 cyber note comes closest.
Leadership statements look similar. Chen found 15 frontier-safety statements from founders, CEOs or chief scientists in nearly four years, and six of the nine that propose something come from Zhipu. DeepSeek's Liang Wenfeng has said nothing.
Who makes the rules
In 102 coded texts, frontier risk appears in 86% of technical scientists' writing but only 22% of law scholars'. Of the 42 technical texts, 52% argue that safety should come before development. No legal scholar or official takes that position. The lawyers' slogan, "failing to develop is the greatest insecurity," comes from Xi's 2014 line that "development is the greatest security." Thirteen texts called for binding frontier duties, such as registering training runs and requiring safety cases before release. None of those demands has been adopted.
China does have evaluation capacity, including Shanghai AI Lab's framework with 13 red lines and CAICT's Benchmark 2.0. Chen argues that none of these bodies can compel anyone to submit a model for testing.
Chen ends with Anthropic: within days of calling for pacing, it shipped Claude Opus 5.5 and described pacing as a way of "remaining competitive with China."
Remember
Of 857 Chinese model releases, 94.9% have no safety disclosure, and only 1.1% had results at launch.
Chinese rules regulate outputs and deployment. None are triggered by compute, unlike the EU's 10²⁵ FLOP or SB 53's 10²⁶ thresholds.
Scientists called for binding frontier rules, while lawyers adapted Xi's development-first doctrine, and the lawyers' view won.
Doodle-in LoRA. This one replaces a magenta scribble with a named object. There was no dataset, so the agent built 6,042 pairs: it removed objects from Open Images photos with LaMa inpainting and drew scribbles where they had been. Checkpoint comparison picked step 500. With a turbo LoRA at 6 steps, 67.5% of objects were detected where they were drawn. Held-out classes did about as well as seen ones (65.0% vs 64.2%). Cost: $24.
Agate 4-step. A 260M text-to-image model that needed 50 guided steps (100 passes per image) was distilled to 4. The agent baked in guidance, then cut steps in stages from 16 to 8 to 4. A second fine-tune on 24,000 teacher pairs raised GenEval from 0.509 to 0.536, against the teacher's 0.563 at 50 steps. Cost: $37.
Huggy LoRA on FLUX.2 klein 4B, trained on 84 drawings. Step 200 was the first fully on-model checkpoint, and from step 500 the style bled into unrelated prompts. Cost: $7.60.
How the prompts work
The author's prompts grew from about 450 words to about 2,000 over the six projects, and all of them are on GitHub. Each one includes:
a section headed "Verified facts, do not re-derive", so the agent doesn't spend budget rediscovering known details
a required zero-shot baseline before any training
a smoke test with a check attached, such as 50 steps and then confirming the weights actually changed
a fixed list of deliverables
a hard spending cap
The agent starts every task with a $0 budget and must ask before running any paid job.
Remember
The 0.8B distilled rewriter gives valid output 99.7% of the time on a CPU, using a quarter of the 9B teacher's tokens, for $16.
Fine-tuning Qwen3.5-2B raised citrus diagnosis accuracy from 14.9% to 52.8% for $1.90 of compute.
The prompts that worked required a baseline, a checked smoke test, a verified-facts section and a hard spending cap.
