Retrieval Forecasts, Faster Macs, Real Clinics · Morning Gist
Monday, October 265 stories · 16 min
Retrieval Forecasts, Faster Macs, Real Clinics
Open tools you can test today, a medical AI that made it into clinics, and benchmark and vendor numbers worth reading with suspicion.
Picked from 45 new articles across 24 sources · arrow keys for other days
Today in 30 seconds
30s left
NVIDIA open-sourced Kumo Forecast under Apache 2.0. It pairs a frozen 340M MOMENT encoder with retrieval over your own history. No benchmarks published yet. story 1 ↓
PyTorch 2.15 makes mixture-of-experts LLMs decode 5–6.5× faster on Macs, adds FP8 on Apple GPUs and cuts one video model's memory from 270 GiB to 8 GiB. story 2 ↓
Google's 2019 tuberculosis X-ray model is now in clinics: 25,000 scans read in six African countries, and Indonesia plans to screen 50 million people a year. story 3 ↓
Kimi K3.1 Pro scores near the top on benchmarks but struggles on long tasks. AI-found bugs are now turning up in PHP and LiteLLM. story 4 ↓
OpenAI customer Terminal X says its GPT-6.1 Sol grader agreed with senior reviewers 88.6% of the time. The sample was only 500 judgments, against an unnamed rival. story 5 ↓
1/5Hugging Face· Oct 21
NVIDIA open-sources Kumo Forecast, a time-series model that retrieves past patterns
NVIDIA released Kumo Forecast under the Apache 2.0 license. The model predicts sensor-style numeric series with a pretrained Transformer, can blend in similar patterns retrieved from a user's past data, and explains its forecasts. The post reports no accuracy numbers.
Why it matters · It applies retrieval augmentation, a familiar idea from LLM work, to numeric forecasting in an open model you can test and fine-tune. You will have to check the claims yourself.
Know this first
Context from the editor, not the article · 31 sec
1 · Time-series foundation models
Instead of training one forecaster per dataset, labs now pretrain large Transformers on many unrelated time series, such as energy, traffic and sensor readings. They then reuse the pretrained model, or a lightly trained head on top of it, for new forecasting tasks.
2 · What MOMENT is
MOMENT is an open family of time-series foundation models from Carnegie Mellon (2024). It splits a series into fixed-length patches and is pretrained to reconstruct masked patches. Its encoder produces general-purpose representations for forecasting, anomaly detection and classification.
3 · Retrieval augmentation
2/5Hugging Face· Oct 20
PyTorch 2.15 makes Mac mixture-of-experts LLMs decode 5–6.5× faster
Hugging Face engineers rewrote slow parts of PyTorch's Apple GPU backend as hand-written Metal kernels. In PyTorch 2.15, mixture-of-experts models generate tokens 5–6.5× faster, and FP8 weights now work on Macs.
Why it matters · If you prototype or serve models on Apple silicon, many slow paths and memory blowups on the Mac's GPU backend are now fixed upstream, so you no longer need to work around them.
Know this first
Context from the editor, not the article · 29 sec
1 · What MPS is
MPS (Metal Performance Shaders) is PyTorch's backend for Apple silicon GPUs. Many of its ops were built on MPSGraph, Apple's general graph framework, which can be slow and wasteful compared with hand-written Metal GPU kernels.
2 · Prefill versus decode
An LLM processes the prompt all at once (prefill), then generates one token at a time (decode). In decode, each step pushes a single row through the model, so matrix multiplies become matrix-vector products that are limited by memory bandwidth.
3 · Mixture-of-experts models
An MoE layer has many expert sub-networks. A router picks a few experts per token (top-k), so each expert gets a different number of tokens. Names like 35B-A3B mean 35B total parameters, about 3B active per token.
3/5Google Research· Oct 23
Google traces its tuberculosis X-ray AI from 2019 paper to deployment in clinics
Google says its tuberculosis-screening AI for chest X-rays now runs in about 40 African clinics through a certified partner product. It is also releasing open models that health ministries, Indonesia's included, can fine-tune on their own X-rays.
Why it matters · It shows how much work sits between a model that matches radiologists on a benchmark and a clinical tool people use: certification, offline use, local fine-tuning and data residency.
Know this first
Context from the editor, not the article · 33 sec
1 · TB and the radiology bottleneck
Tuberculosis is curable, yet it kills over a million people a year. Screening usually starts with a chest X-ray. Where TB is common, there are few radiologists to read those X-rays, so results can take weeks and infected people keep spreading the disease.
2 · Screening versus diagnosis
Screening tools are tuned for high sensitivity, so they miss few true cases even if that means more false alarms. Anyone flagged gets a confirmatory molecular test, such as a sputum PCR. The AI only triages; it does not give the final diagnosis.
3 · Embeddings and fine-tuning
4/5Interconnects (Nathan Lambert)· Nathan Lambert· Oct 25
Kimi K3.1 Pro scores near the top but long tasks expose gaps
Kimi's new 3-trillion-parameter open model ranks about fourth on a major benchmark index but trails on long, exploratory tasks. Two researchers argue the recent wave of AI-found vulnerabilities isn't really about open versus closed models.
Why it matters · Choosing an open model by its headline benchmark score can mislead, and AI-found security bugs are now hitting software stacks you probably depend on, including PHP and LiteLLM.
Know this first
Context from the editor, not the article · 28 sec
1 · Open-weight versus closed models
Open-weight models publish their trained parameters, so anyone can run or modify them. Closed models are served only through the lab's API, so the lab can update safeguards at any time and see who is using them.
2 · How post-training shapes behavior
After pretraining, labs run a mid-training stage on curated data, then reinforcement learning (RL) on tasks with checkable rewards. Heavy RL on narrow, goal-directed tasks can hurt a model's ability to explore open-ended problems.
3 · AI and vulnerability hunting
5/5OpenAI· Oct 20
Terminal X's GPT-6.1 Sol grader agrees with senior reviewers 88.6% of time
Terminal X, which builds AI agents for investment firms, uses OpenAI's GPT-6.1 Sol to grade those agents' work. On 500 judgments, its pass/fail verdicts matched a senior human reviewer 88.6% of the time, against 86.4% for the next-best model.
Why it matters · It is a concrete production example of a model grading another model's output against a rubric, with its own numbers, and it shows how thin vendor evidence for model choice can be.
Know this first
Context from the editor, not the article · 31 sec
1 · Model as grader
Teams often use one language model to score another's output against a rubric or reference answer, because human review doesn't scale. The grader is trusted only as far as it agrees with expert humans, so that agreement rate is the key number.
2 · Investment memos and mandates
Funds write memos to justify each deal to an investment committee. Every fund has a mandate, the kinds of deals it is allowed and meant to make. A recommendation outside that mandate is a mistake, even if the deal itself looks good.
3 · Vendor customer stories
This piece was published by OpenAI about one of its customers. Comparison numbers in posts like this usually come from the customer's own internal evals. The competing models are often unnamed and the methods unpublished.
Retrieval augmentation means looking up stored examples similar to the current input and feeding them into the prediction. LLMs do this with documents. Here the stored examples are windows of historical numeric data, matched by nearest-neighbor search.
FOX Blueprint
NVIDIA's Factory Operations reference design; its predictive-maintenance module uses Kumo Forecast and Kumo Anomaly.
Image: Hugging Face
What shipped
NVIDIA has released Kumo Forecast as open source under Apache 2.0. It was previously called NV-Tesseract Forecasting. The model forecasts 1, meaning several numeric columns that share a timestamp, such as sensor readings from a machine. The code is in NVIDIA/Kumo-TS on GitHub and the weights are in nvidia/Kumo-Forecast on Hugging Face.
Be clear about what the announcement contains: no accuracy benchmarks, no comparisons with other models, and no measurement of how much its main feature helps. The method details are in two separate papers.
The idea
A forecaster can lean on general patterns learned in pretraining, or on the specific history of the system it is predicting. Kumo Forecast uses both.
Direct path. The backbone is 2, an open time-series Transformer from Carnegie Mellon, pretrained to reconstruct masked chunks of many kinds of series. NVIDIA puts the encoder at about 340 million parameters. It stays frozen, and only a small forecasting head on top is trained. When you ask for a horizon longer than the head natively produces, the library extends the forecast 3, feeding its own predictions back in. Errors can compound when it does this.
Retrieval path (4). In DARR mode (Domain-Aware Representation and Retrieval), you pass a separate historical DataFrame, context_df. The model runs a 5 search to find the past windows most similar to the current input, then blends what it finds with the direct forecast. A setting controls the weight given to each path. A schematic example: if a motor's current vibration pattern looks like episodes from last year that came before overheating, the forecast leans toward that outcome. The post doesn't say how similarity or blending are computed.
Explanations. Optionally, each forecast comes with attributions for every forecast step, showing which past time steps and which input columns mattered. It also includes diagnostics on the model's internal representations and, optionally, 6, a standard gradient-based attribution method. The forecaster itself is left unchanged.
Constraints
Inputs must be numeric and exactly 256, 512, 1024 or 2048 steps long.
The retrieval store must have dimensions similar to the input.
Where it's used and how to judge it
NVIDIA uses the model for predictive maintenance inside its Factory Operations (FOX) Blueprint, paired with a sibling model, Kumo Anomaly. The repository includes an example of fine-tuning on a user-supplied CSV.
NVIDIA's advice is to run direct mode and DARR mode on the same held-out windows across the horizons you need. Without published numbers, that comparison is the only evidence you'll have about whether retrieval helps your data.
What shipped
NVIDIA has released Kumo Forecast as open source under Apache 2.0. The model was previously called NV-Tesseract Forecasting. It forecasts 1, meaning series with several numeric columns that share a timestamp. The code is on GitHub as NVIDIA/Kumo-TS and the weights are on Hugging Face as nvidia/Kumo-Forecast.
The model has three parts:
a pretrained Transformer that forecasts directly;
an optional retrieval mode called 2 (Domain-Aware Representation and Retrieval), which mixes in patterns pulled from your own historical data;
an interpretability layer that explains each forecast.
The announcement contains no accuracy numbers. There are no benchmark tables, no comparison with other forecasting models, and no ablation showing how much retrieval helps. It points to two separate papers for the method details. Read it as a release note, not as evidence that the model works well.
The problem it addresses
A forecaster can draw on two kinds of knowledge. One is general temporal structure learned in pretraining: trends, seasonality, how a signal typically behaves after a spike. The other is the specific history of the system being forecast. A factory pump may have failed in a recognizable way three times last year, and a generic pretrained model has never seen those episodes.
Fine-tuning is one way to bring that local knowledge in, but it means retraining whenever the history changes. Retrieval is another way. You keep the history around as a searchable store and look it up at prediction time. Kumo Forecast supports both approaches.
How it works
Backbone. The encoder is 3, a Transformer pretrained on a broad mix of time series. It splits a series into patches and learns to reconstruct masked ones. NVIDIA puts the base encoder at roughly 340 million parameters. The encoder and embedding weights stay frozen. Only a forecasting head on top is trained. This keeps the general-purpose representation separate from the task-specific prediction layer, and adapting the model costs about as much as training a small head.
Direct forecasting. The head maps the encoder's representation of the input window to future values over a chosen horizon. If you ask for a horizon longer than the head natively produces, the library extends the forecast 4: it feeds its own predictions back in as input and predicts again. The post treats a directly generated forecast and a recursively extended one as different things, and that matters in practice. Recursive forecasts compound their own errors, so accuracy beyond the native horizon is likely to fall off faster.
DARR mode. You pass a second DataFrame, context_df, holding historical data. The model runs a 5 search, finding the k stored windows most similar to the current input, and blends what it retrieves with the direct forecast. A configuration option sets how much weight each pathway gets.
As a schematic example (not from the article): suppose the input window is the last 512 readings of vibration and temperature from a motor. The direct path produces a 96-step forecast from pretrained knowledge. DARR finds, say, the five most similar 512-step windows in two years of that motor's logs. It looks at what followed each one and folds those outcomes into the prediction. If similar patterns came before a temperature climb in the past, the final forecast leans that way.
The post separates the input window (recent observations) from the retrieval context (the historical store). The article does not say how retrieved neighbors are represented, which similarity metric is used, or exactly how the blending works. For that it points to the paper Memory-Augmented Forecasting: Scalability and Generalization Across Temporal Domains.
Interpretability. When explanations are turned on, each forecast comes with an "explanation bundle" containing:
lag and feature attributions for every forecast horizon, showing which past time steps and which input columns drove the prediction for step 1 versus step 96;
"semantic-flow" and "trajectory-stability" diagnostics, which look at how the model's internal representation moves;
optional 6 on the embeddings. Integrated Gradients is a standard attribution method that adds up gradients along a path from a baseline input to the real one.
The framework is described as model-agnostic, and it does not change the forecaster. The method comes from a companion paper, NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates. The post does not define the two diagnostics in any more detail.
Constraints
Input must be numeric, with a timestamp column and one or more value columns. It can arrive as a pandas DataFrame, CSV or JSON.
Input length must be exactly 256, 512, 1024 or 2048 observations.
In DARR mode, the historical context must have dimensions similar to the input. You cannot retrieve from a store with different columns.
Deployment and tooling
NVIDIA's Factory Operations (FOX) Blueprint, a reference design for factory systems, includes an early-access Predictive Maintenance module. It combines Kumo Forecast with a sibling model, Kumo Anomaly, to predict anomalies from live data. That is the use case NVIDIA has in mind: industrial sensor streams with long histories. The repository also includes an example of fine-tuning on a user-supplied CSV.
What to make of it
The design is reasonable. A frozen foundation encoder plus a cheap head, with retrieval bolted on, lets you add local history without retraining. NVIDIA's own advice is the right test: run direct mode and DARR mode on the same held-out windows, across the horizons and operating conditions you care about.
Several questions remain open:
How much does retrieval actually improve accuracy, and on which kinds of series?
How does retrieval latency grow with the size of context_df?
How does a frozen MOMENT backbone compare with other open time-series models?
The post answers none of these. The papers it cites might.
Remember
The MOMENT encoder stays frozen and only the forecasting head is trained, so adapting the model is cheap.
Retrieval adds local history at prediction time without retraining, but the post never measures how much it helps.
Inputs must be exactly 256, 512, 1024 or 2048 steps; longer horizons use recursive, error-compounding extension.
The grouped-matmul decode kernel alone gives about 1.3× more tokens/s on an M3 Ultra for two MoE models.1 / 3
PyTorch 2.15 brings large speedups to 1, PyTorch's backend for Apple silicon GPUs. The work comes from Hugging Face's PyTorch team. Their method is simple: run real models from the Hugging Face Hub on a Mac, find where they are slow or break, and fix the problem upstream. More and more often, the fix replaces 2, Apple's general-purpose graph framework, with a hand-written 3 kernel.
Mixture-of-experts decoding
The headline number is for 4 (MoE) models. These models send each token to a few expert sub-networks. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B went from 9.4 tokens/s on 2.14 to 61.3 tokens/s. LFM2.5-8B-A1B went from 23.6 to 123.6 tokens/s.
Most of the gain comes from 5, which multiplies uneven groups of tokens by their own expert weights in a single operation. In 2.14, PyTorch copied the group boundaries to the CPU and looped over the groups. Now Metal kernels read the boundaries directly on the GPU, and a dedicated kernel handles the small groups typical of decoding. That decoding kernel alone accounts for about 1.3× (38.7 → 51.7 tokens/s for Qwen at 512 context). Fewer GPU-CPU syncs and faster top-k routing make up the rest. The post doesn't break down exactly how each piece contributes.
Every LLM gets something
During decode, each new token passes through the model as a single row. F.linear now uses 6 (matrix-vector) kernels for that case. Across 96 real BF16 layer shapes, this gives a 1.23× geometric-mean speedup. Most of it shows up on small projections such as MoE gates. Large layers were already limited by memory bandwidth, so they stay near 1×. Softmax over the vocabulary, used when sampling the next token, is about 8× faster: 57 µs down to 7 µs for a 128,256-entry row.
7 and memory fixes
float8_e4m3fn tensors now work on MPS. They store one byte per value, half the space of BF16, and F.scaled_mm multiplies them with per-tensor or per-row scales.
Moving constant padding from MPSGraph to Metal had an outsized effect. The Wan2.2 video model's tiled 8 decode makes 8,764 padding calls. With the old path, peak driver memory reached about 270 GiB. With the new one it is about 8 GiB. The same change fixed silent image corruption in the Qwen-Image-2.1 encoder: round-trip quality rose from 10.06 dB to 29.95 dB 9, matching the CPU result.
Requirements and what's next
MPS now needs macOS 15 or later, and PyTorch 2.15 needs Python 3.11 or later. For 2.16, the team previews attention running entirely on Metal (one decode case drops from 465 µs to 65 µs on an M5 Pro). It also previews sending GPU work in batches of 16 kernels, which speeds decoding by 4–19%. Both are still subject to review.
PyTorch 2.15 includes a large batch of improvements to 1 (2 Performance Shaders), the backend that runs PyTorch on Apple silicon GPUs. Hugging Face's PyTorch team did much of the work, together with PyTorch maintainers and community contributors. The team says that at the start of 2026, even small models on MPS often hit unsupported operations, sudden performance drops and unexplained memory growth. Their approach is to run real models from the Hugging Face Hub (language, image, video), find the failures, and fix them upstream. More and more often, the fix replaces an 3 implementation, built on Apple's general graph framework, with a dedicated Metal kernel.
Why decoding is the target
LLM inference has two phases. Prefill processes the whole prompt at once. Decode then generates one token at a time. During decode, each new token passes through the model as a single row, so the large matrix-matrix multiplies turn into matrix-vector products. These are limited by memory bandwidth: the GPU spends its time reading weights, not doing arithmetic. Any per-call overhead also shows up directly, because it repeats for every token.
Dense linear layers. MPS has had 4 (general matrix-vector multiply) kernels for torch.mm since 2.14, but F.linear, the call that most models actually use, still went through MPSGraph. Version 2.15 sends the common [1, 1, hidden_size] decode input to the GEMV kernels and fuses eligible bias additions into them. Across 96 BF16 layer shapes taken from real models, the geometric-mean speedup is 1.23×. The gains concentrate on small projections, such as MoE gate layers, where the old path left most of the memory bandwidth unused. Large layers and LM heads were already close to the bandwidth limit, so they stay near 1×. That is what you would expect if the fix removes overhead rather than adding compute.
Mixture-of-experts layers. A 5 (MoE) layer routes each token to a few of many experts, so the experts receive uneven numbers of tokens. Grouped matmul handles this in one call. You pass all tokens sorted by expert, plus an offsets tensor marking where each group ends. Take the article's worked example: 64 tokens split into groups of 16, 32 and 16, with offs=[16, 48, 64]. Each group is multiplied by its own expert's weight matrix. In 2.14, MPS copied those offsets to the CPU and looped over the groups in Python-side code. That forced a GPU-CPU synchronisation and launched one operation per expert. The new Metal kernels read the offsets directly on the GPU. They use Metal Performance Primitives (Apple's optimised GPU building blocks) where available, a separate Metal path elsewhere, and a dedicated kernel for the small groups typical of decoding.
The MoE numbers
On an M3 Ultra with torch.compile, the release's end-to-end numbers compared with 2.14 are:
Qwen3.6-35B-A3B: 9.4 → 61.3 tokens/s
LFM2.5-8B-A1B: 23.6 → 123.6 tokens/s
That is 5–6.5× overall. The team attributes it to three changes: native 6 with its decode path, fewer GPU-CPU syncs, and faster top-k routing. The chart that isolates the decoding kernel shows a smaller step of about 1.3×:
Qwen: 38.7 → 51.7 tokens/s at 512 context, 36.8 → 48.7 at 2048
LFM2.5: 77.1 → 103.2 and 74.9 → 98.7
So most of the 5–6.5× apparently comes from removing the CPU round trip and the other changes, not from the specialised small-group kernel. The article doesn't fully reconcile these figures. For example, it doesn't say which configuration produces 61.3 tokens/s.
Sampling.softmax and log_softmax now run on dedicated Metal kernels. In FP32 over a single [1, 128256] row of logits, softmax takes 7 µs instead of 57 µs. Across vocabularies from 128,256 to 201,088 entries, the speedup is 7.1–8.0× (50–74 µs down to 7–10 µs).
7 on the Mac
torch.float8_e4m3fn tensors now work on MPS. They store one byte per value, half the size of FP16 or BF16. F.scaled_mm multiplies them with tensorwise or rowwise scaling and can output BF16. The usual recipe is to divide a tensor by amax / finfo.max, cast it to FP8, and pass the scales to the matmul. The article frames this mainly as a memory saving. It notes that native FP8 matrix multiplication in hardware waits on Metal 4.1 shaders, so the current gain is storage, not FP8 compute speed.
Padding and a 270 GiB surprise
Constant padding moved from MPSGraph to Metal, and that turned out to matter a great deal. The tiled 8 decode in Wan2.2-TI2V-5B makes 8,764 padding calls across 28 tiles. Peak memory before and after:
Tensor allocations: 4.42 GiB → 4.10 GiB
Allocator-reserved memory: 8.07 GiB in both cases
Total driver memory: 270.1 GiB → 8.1 GiB
About 262 GiB of the old total sat outside PyTorch's allocator, so PyTorch's own memory accounting would never have shown it. The post doesn't explain why the MPSGraph path grew driver memory this way. The same change fixed a correctness bug: the Qwen-Image-2.1 VAE encoder had been silently corrupting images on MPS, which broke ComfyUI edit workflows. An encode-decode round trip now matches CPU at 29.95 dB 9 (peak signal-to-noise ratio), up from 10.06 dB.
Other changes
Linear algebra: complex64 LU factorisation, LU solve and triangular solve now run on MPS, which also enables complex determinants and their gradients. A single kernel inverts batches of FP32 matrices up to 4×4. QR uses blocked updates. Gated DeltaNet, a linear-attention architecture, gets a fast path for batches of 16/32/64-size triangular systems with at least eight right-hand sides. SVD handles numerical extremes more reliably.
Convolutions and training: specialised conv1d kernels for audio shapes; Metal reflection and replication padding, with backward-pass correctness fixes; dropout through the fused kernel; a Metal upsampling backward; channels-last conv2d backward that stays channels-last.
Streams:torch.mps.Stream, a torch.mps.stream context manager and Tensor.record_stream give Python control over which stream runs which work.
New ops:frexp, logspace, take/put_, heaviside and others. There is also a broadcasting fix for the math implementation of scaled dot-product attention.
Requirements: macOS 15+ and Python 3.11+.
Previewed for 2.16 (not final)
Once a decode path contains no MPSGraph ops, the GPU waits until the CPU has encoded every kernel for a token. Version 2.16 commits the command buffer every 16 kernels, so the GPU can start earlier. Compiled BF16 decoding at 2048 context gains 16% on LFM2.5, 19% on Qwen3.6-35B-A3B and 4% on the dense Qwen3.8-27B.
Scaled dot-product attention is moving entirely to Metal. On an M5 Pro, a grouped-query attention decode (32 query heads, 8 KV heads, 4K context) drops from 465 µs to 65 µs. A 4K × 4K prefill with head dim 40 drops from 42 ms to 6 ms.
bincount on 4 million skewed integers falls from 6.8 s to 0.5 ms. Building shaders for Metal 3.1 by default shrinks libtorch_cpu.dylib from 384 MiB to 287 MiB.
The pattern across the release is consistent. Many of MPS's problems were not about hardware limits but about MPSGraph overhead, CPU round trips and hidden memory use, and each fix is a targeted kernel. The open question is how much of PyTorch's op surface still needs that treatment.
Remember
Most of the MoE speedup came from removing GPU-CPU round trips; the new decode kernel alone added about 1.3×.
Swapping MPSGraph padding for Metal removed about 262 GiB of driver memory that PyTorch's allocator never counted.
FP8 on Mac currently saves memory; native FP8 matmul speed waits on future Metal 4.1 shaders.
A foundation model turns an image into an embedding, a compact numeric summary. Local teams can train a small classifier on those embeddings, or fine-tune the whole model, using a few thousand of their own labelled scans instead of training from scratch.
Image: Google Research
Google Research used 1, an infectious-disease conference, to give a progress report on its tuberculosis (TB) work. The post covers a chest X-ray model that is now a certified product in African clinics and a set of open models that health systems can adapt themselves. It is mainly a program overview, so it has few new evaluation numbers.
The problem
TB kills more than 1.23 million people a year. Screening starts with a chest X-ray (2), and in high-burden areas there are few radiologists to read them. Results can take weeks, and cases go undetected.
The evidence trail
2019, Radiology: Google's deep-learning model detected active pulmonary TB as well as experienced radiologists did.
2021, The Lancet Digital Health: In an independent evaluation of computer-aided detection (3) software, the Stop TB Partnership ranked Google's model among the top performers. WHO recommended CAD for TB screening the same year.
2024, NEJM AI: A prospective, multi-site study found that sensitivity stayed high across patient groups and that the model also caught other lung and heart abnormalities. The post does not give the figures.
Crossing the "translation gap"
Google uses this term for the work between a published model and a tool clinicians trust. It covers reliability across X-ray machines, offline operation, workflow integration and regulation. Nexus AI, a medical-technology company in Pretoria, turned the model into a product. That product has EU 4 medical-device certification and 5 quality certification, and Nexus AI is pursuing WHO prequalification and US FDA clearance.
The tool runs in about 40 facilities across six countries. More than 90 clinicians have used it on over 25,000 X-rays, and turnaround has dropped from weeks to under 45 seconds. At the Mponeng gold mine in South Africa, portable X-ray kits run the AI offline 4 km underground. Screening, triage and referral for a confirmatory test now happen in one visit.
Open models instead of one product
Google decided that a single partner product cannot meet global demand. It has open-sourced:
Chest X-ray embeddings and 6, a small model that links medical images to clinical text. It can classify images using plain-language labels, or be fine-tuned with a few local examples.
7, open-weight multimodal models with 4B and 27B parameters. MedGemma combines X-ray findings with patient history, answers clinicians' questions and runs offline. It has passed 10 million downloads.
Indonesia has the world's second-highest TB burden and wants to screen 50 million people a year. Its health ministry is fine-tuning both models on Indonesian X-rays and running them on its own infrastructure, so patient data stays in the country.
For places with no X-ray machine, Google's 8 (Health Acoustic Representations) is an audio model trained on 300 million clips. It screens for TB from coughs recorded on a phone. Studies are underway in Zambia and Ghana, but the post reports no accuracy results for cough triage or for the Indonesian models.
At 1, an infectious-disease conference in Washington, D.C., Google Research gave an overview of its tuberculosis (TB) screening work. It covers a deep-learning model for chest X-rays that a partner company has turned into a certified clinical product, and a later move toward open models that national health systems can adapt and run themselves. The post is a program overview, not a paper. It summarizes earlier published evidence and adds deployment numbers, but it reports no new accuracy figures.
The problem
TB is preventable and curable, yet it kills more than 1.23 million people a year. Google names radiology as the main bottleneck. Chest radiography (2) is the first-line imaging test for lung disease. Spotting subtle TB signs takes expertise rural clinics rarely have. Two examples are apical infiltrates, meaning fluid in the top of the lung, and cavitary lesions, meaning cavities where tissue has been destroyed. Where radiologists are scarce or imaging is days away, cases go undetected and people keep transmitting the disease.
Screening is a triage step. The goal is high sensitivity, so few true cases are missed. Anyone flagged as "presumptive TB" goes on to a confirmatory molecular test. In this setup the AI decides who needs follow-up. It does not make the diagnosis, and clinicians make the final call.
The evidence trail
Google lists a sequence of results that build on each other:
2019, Radiology. A retrospective benchmark showed deep-learning models could detect active pulmonary TB as well as experienced radiologists.
August 2021, The Lancet Digital Health. The Stop TB Partnership, a UN-hosted coalition, independently tested computer-aided detection () products on varied patient groups and ranked Google's model among the best in the world. That year, the World Health Organization (WHO) recommended CAD for TB screening and triage. This matters because procurement in high-burden countries often depends on WHO guidance.
The order matters. A retrospective benchmark, an independent head-to-head comparison and then a prospective study are the steps that turn a model claim into clinical evidence. The post gives no sensitivity, specificity or AUC values from any of the three studies, so a reader who wants the numbers has to go to the papers.
The translation gap
Google calls the distance between a published model and a trusted bedside tool the "translation gap." Closing it means running reliably on many different X-ray machines, working offline, fitting into clinical workflows and passing medical-device regulation. Its partner Nexus AI, a medical-technology company in Pretoria, did this work. The product has:
CE MDR 4 certification under the EU Medical Device Regulation (2017/745). Class IIb is a higher-risk tier that requires clinical evaluation, here based on prospective data.
5 certification, the international quality-management standard for medical-device makers.
An agreement with FIND, a global diagnostics nonprofit, to seek , which UN and global agencies require before they buy a product. Nexus AI is also working toward US FDA clearance.
Deployment numbers
The tool runs in about 40 facilities in South Africa, Sierra Leone and Côte d'Ivoire, with expansion underway in the DRC, Zambia and Cameroon. More than 90 clinicians have used it on over 25,000 chest X-rays. Turnaround has gone from weeks to under 45 seconds. Nexus AI has also provided more than 3,000 free screenings to public clinics and aims for 10,000 sponsored screenings a year.
The most extreme site is the Mponeng gold mine in South Africa, one of the deepest in the world. Miners there breathe silica dust, which raises their risk of lung disease. Ultra-portable digital X-ray kits running the AI offline operate 4 km underground. Miners flagged for presumptive TB are sent straight to confirmatory molecular testing. Screening, triage and testing now happen in one visit instead of over several weeks. The post does not say how many miners have been screened or how many cases were confirmed.
From one product to open models
Google argues that a custom product with one partner cannot cover the global TB burden, so it now releases building blocks through Health AI Developer Foundations (7):
Chest X-ray embeddings, compact numeric summaries of a scan that a small classifier can be trained on.
8, a lightweight vision-language model trained so that medical images and the clinical words describing them land close together in a shared representation. Developers can therefore classify or search X-rays with plain-text labels without training, or fine-tune on a small local dataset. An illustrative case: a regional team with a few thousand labelled local scans could get a usable TB classifier without large compute budgets.
Indonesia
Indonesia has the world's second-highest TB burden and wants to screen 50 million citizens a year. Reading that many X-rays by hand is not possible. Proprietary cloud software would also raise data-residency problems, because patient images would leave the country. The Ministry of Health is instead fine-tuning MedGemma and MedSigLIP on Indonesian chest X-rays and running them on local infrastructure. Google's case for open weights is about sovereignty as well as cost: the ministry controls the model, the data and the deployment. The post does not report how well the fine-tuned models perform or how far the rollout has got.
Cough screening with 10
Some health posts will never have an X-ray machine. For them, Google built Health Acoustic Representations (HeAR), an audio model trained on more than 300 million audio segments. It screens for TB from coughs recorded on an ordinary smartphone. Research partners include CIDRZ in Zambia, the Aurum Institute in Ghana (supported by a $250,000 Google.org grant) and a $1.5 million regional collaborative run by HELINA. Google says these groups are still "building the clinical evidence," and the post gives no accuracy data.
Google presents the three tools as one diagnostic cascade. Cough triage with HeAR comes first, then mobile X-ray screening with HAI-DEF models, then clinical reasoning with MedGemma.
What to take from it
The useful lesson for practitioners is the shape of the path, not any single metric. It ran from a retrospective benchmark to independent comparison, prospective validation, a regulated product built by a local partner, offline hardware for difficult sites, and finally open weights so health systems can fine-tune and host models themselves. Several questions stay open. The post gives no performance numbers, no evidence that MedGemma's reasoning is safe for TB triage specifically, no results from Indonesia's fine-tuning and no validation of cough-based screening. The 25,000 X-rays and 40 facilities are real usage figures, but they measure reach, not outcomes such as cases found or deaths prevented.
Remember
Google's evidence came in stages: a retrospective benchmark, then an independent comparison, then a prospective multi-site study, before any product existed.
Open weights let Indonesia fine-tune on its own X-rays and keep patient data inside the country.
Deployment figures (25,000 X-rays, under 45 seconds) show reach; the post gives no accuracy or outcome numbers.
In recent months, frontier models have found software vulnerabilities at scale. OpenAI and Anthropic give vetted defenders access to these abilities through "trusted access" programs, which has revived the argument over whether open models are dangerous.
CTF
Capture-the-flag: hacking puzzles used to train and test offensive security skills.
Image: Interconnects (Nathan Lambert)
A top-four model with a long tail of doubts
Kimi K3.1 Pro, a roughly 3-trillion-parameter open-weight model from Moonshot, ranks about fourth on the 1, a widely followed benchmark aggregate. Only the two latest Claude 5.5 models and Meta's 2 Wonder Full score higher, and it beats OpenAI's GPT-6 Astra. Yet on the Interconnects podcast, Nathan Lambert and evaluation researcher Florian Brand describe a mixed reception. Moonshot's own blog post shows it about 20 points behind GPT-6 Astra on Terminal-Bench Science and weaker on 3.
Brand's explanation: the Artificial Analysis tasks are short and goal-directed, while ProgramBench is long and exploratory. He suspects Kimi's 4 rewarded pushing toward known targets over exploring. Lambert argues that is exactly the wrong weakness for a model this large. For well-defined coding work, smaller models such as DeepSeek-V4.1-Flash come close and run much faster. Serving is awkward as well: one copy of the model needs a full node of Nvidia 5 GPUs. Weights are due by November 4, after about two weeks of API-only access.
Weaker at hacking, on purpose
Kimi says it won't release open weights that "extend the absolute frontier of cyber capability." It down-weighted cyber data during 6, the stage between pretraining and fine-tuning. Brand argues that finding and fixing bugs doesn't need long-horizon skill, while breaking into a whole organization does. So cutting offensive training gives up little defensive value.
The cyber picture
Lambert says security experts see recent events as a continuation of trends that were exponential before ChatGPT. AI people, by contrast, frame them as an open-versus-closed fight. The episode cites three cases:
PHP. A long-standing PHP bug that let attackers run code on servers was found and fixed in a coordinated effort involving OpenAI's Patch the Planet program. There are reports that mass exploitation hasn't followed.
7. The popular library was compromised through an API key that had sat exposed on GitHub for five months.
Old systems. Some small businesses run PHP versions so old they no longer get security patches at all.
The hosts' reading is that there are few attackers, so rising capability doesn't scale attacks tenfold. Defense is the bigger worry, because coding agents are producing code faster than the existing backlog of flaws can be fixed. Lambert credits closed labs' "8" programs, now easier to join, for many of the disclosures. He criticizes companies that serve "9" models, meaning models with their refusals removed, through an API. He concedes his earlier "open-weight Mythos would've been fine" take didn't account for vulnerability trends that were predictable.
The episode also covers Anthropic's decision to stop hiring junior employees, which Lambert calls a cultural red flag, and Brand's surprisingly good experience using Meta's Muse agent for errands.
The release
Moonshot's Kimi K3.1 Pro is a roughly 3-trillion-parameter model. On the 1, a widely watched aggregate of benchmark scores, it ranks about fourth. Only the two latest Claude 5.5 models and Meta's 2 Wonder Full score higher, and it beats OpenAI's GPT-6 Astra. Even so, reactions have been mixed. In an Interconnects podcast, Nathan Lambert and Florian Brand, an evaluation researcher, try to explain why. They then spend most of the episode on cybersecurity.
The launch itself followed a pattern Brand calls the "professionalization" of model releases. A year ago, open models often shipped before inference engines such as vLLM could run them. Now the inference frameworks and hosting providers support a model on day one. Increasingly, the lab also keeps a stretch of exclusive API access before publishing the weights. For K3.1 Pro that window is about two weeks, with weights promised by November 4. The launch also drew criticism. A screenshot circulated of press instructions telling promoters not to disclose that their posts were paid. Lambert notes that nobody has independently confirmed it.
Why the scores and the impressions disagree
The model's own blog post shows its weaker spots further down the page. It trails GPT-6 Astra by about 20 points on Terminal-Bench Science, and it is also weaker on 3. Brand's explanation is that the Artificial Analysis benchmarks are mostly short tasks with a clear goal. ProgramBench is long and exploratory: the model is told roughly what's wanted and has to work out how to get there. His guess is that Kimi's RL training leans toward exploitation, meaning pushing hard toward a known target, rather than exploration. Lambert puts it more bluntly: the model may be "a tiny bit RL fried", meaning over-optimized by reinforcement learning.
Lambert argues this matters more for a model this large. Size makes it slower and more expensive to serve. For well-defined coding work, a smaller model such as DeepSeek-V4.1-Flash scores not far behind, runs much faster, and may cost about the same per task once token counts are included. A huge model earns its place on vague, open-ended work, which is exactly where K3.1 Pro looks weak. Serving is also awkward. Brand says one copy of the model needs a full node of Nvidia 4 GPUs, while 700-billion-parameter models fit on the more common H200s. The new "Pro" name hints that Kimi may release a smaller "Flash" version later.
Deliberately weaker at hacking
The post also lays out Kimi's "open-weight principles": "we aim not to release open weights that would extend the absolute frontier of cyber capability." According to the hosts, Moonshot down-weighted cyber data during 5, the stage between pretraining and post-training, and found the reduction lasted through post-training. Lambert asks whether this could explain the weak long-horizon scores. Brand thinks the two can be separated. Security professionals tell him that finding and patching bugs doesn't require long-horizon ability, while compromising a whole organization does: an attacker has to keep going through every stage of an intrusion. Closed models, he says, are far ahead there. So leaving long-horizon offensive training and CTF-style (capture-the-flag) hacking puzzles out of the mix keeps the defensive value and gives up little.
"You can hack everything"
Lambert sees two camps that rarely talk to each other. Security experts look at the whole ecosystem. Metrics such as vulnerability discovery and time-to-exploit were already climbing exponentially before ChatGPT, with no clear jump when AI arrived. AI people, meanwhile, argue about open versus closed models and about Mythos, a closed frontier model that was presented in April as a step change in cyber capability. Lambert criticizes AI safety voices for a "boy who cried wolf" pattern of warnings, while granting that vulnerabilities really are being found in bulk.
Three recent incidents anchor the discussion:
PHP remote code execution. A long-standing bug in PHP, the language behind WordPress and much of the web, let attackers run code on servers. OpenAI's Patch the Planet program and other researchers found it and coordinated a fix. Many sites went offline to patch. Brand says there are numerous reports that mass exploitation hasn't followed. Closed-model providers also quickly updated their safeguards to block requests for the exploit.
6. This library, which gives one endpoint for calling many model providers, was compromised again. A preliminary report says an API key had been visible on GitHub for five months before someone used it to spread malware. To Brand, that long gap suggests there are few active attackers.
Lambert's synthesis is that attackers' capabilities rising doesn't multiply attacks tenfold, because attackers themselves are the bottleneck. He cites security writer Joshua Saxe's point that the hacker "supply chain" is thin. Defense is the bigger worry. Following Azeem Azhar, Lambert expects a few years of rising vulnerabilities as coding agents pump out code faster than the existing backlog can be cleaned up.
Access, abliteration and the politics
Brand argues that early demand for open models for security work came partly from how hard it was to get into closed "7" programs. OpenAI's defensive program now just asks for ID verification and hardware security keys. Anthropic is widening access too. Lambert gives these programs much of the credit for the flood of disclosures. He criticizes Abliteration.ai and Morph for serving "8" models, meaning models with their refusal behavior surgically removed, over an API. Defenders mostly want models running on their own infrastructure anyway. Lambert also concedes that his earlier claim that "open-weight Mythos would have been fine" didn't account for vulnerability trends that were predictable. He still expects political trouble for open models, especially if a Chinese-model "swarm" ever hacks a US company.
Side topics
Lambert calls Anthropic's decision to stop hiring junior employees a red flag about its culture. Brand says it leaves more good junior talent for everyone else. Brand also describes using Meta's Muse agent after moving to the US. It proactively tells him which forms he needs for appointments, and it shops for him, checking out through single-use Stripe card numbers. On the downside, it's slow, and its browser sometimes crashes.
Open questions
None of the benchmark explanations here have been tested. They are informed guesses. Whether the AI-found vulnerability backlog turns into real damage, which security people measure mainly through reported ransomware losses, is something both hosts expect to learn over the next few months.
Remember
Aggregate benchmark rankings overweight short, goal-directed tasks; check long-horizon, exploratory evals before choosing a large model.
Kimi deliberately removed cyber data during training, betting that defenders don't need the long-horizon skills attackers need.
The LiteLLM breach came from an API key left public for five months: old hygiene failures still dominate.
How Capital Memory links a firm's judgment and research to the data room and its workflows.1 / 2
OpenAI has published a customer story about Terminal X, a company that builds AI agents for investment firms. The headline figure: Terminal X's grading agent, running on GPT-6.1 Sol, agreed with a senior human reviewer on 88.6% of 500 scored judgments. The next-best model, which isn't named, scored 86.4%.
What's being graded
Terminal X sends engineers who used to work in finance to spend weeks inside a client firm. They study the memos the firm trusts and write down unwritten habits, such as how the credit desk screens a borrower. Then they encode these into hundreds of agents, split by asset class and investment stage. GPT-6.1 Sol handles the stages that need judgment and long context: drafting the investment thesis and checking fit with the fund's 1, meaning the kinds of deals it is supposed to make. The cheaper GPT-6 Luna handles high-volume steps. What the agents learn is stored as a "2", a versioned record of one desk's past analyses, committee decisions and 3.
How the grader works
The Eval Agent reads every agent output against a 54-criterion rubric, much as a vice-president reviews an analyst's work. Does each number, comparable company or quote come from the right source? Does the story hold together? Does the recommendation match the firm's view? Each check is pass or fail. A fail comes with a one-line reason and a correction, and it becomes a new workflow instruction or test case for that desk. Analysts' corrections feed back the same way.
Sol's lower token cost let Terminal X build a deeper review for the same budget. The thesis is checked against the mandate and then against recent deals. Uncertain verdicts are re-run and the results reconciled.
Claimed results
About 25% more catches of 4: recommendations that wander outside the mandate, such as a buyout fund's agent proposing distressed-debt plays when the market looks like turning.
About 30% lower cost per query than the next-best model, at similar speed.
1,700 firm Memories built, more than 1 million workflows run, and 81% of monthly users active weekly. One private equity firm reported $2.4 million a year in savings.
How much to trust it
Take these as vendor claims. With 500 samples, the 88.6% figure carries a margin of error of roughly ±1.4 points (my estimate), so a 2.2-point lead is modest. The article doesn't say whether both models graded the same items. It also doesn't say how often two senior humans agree with each other, which is the real ceiling, or how many drift cases the 25% is based on.
OpenAI has published a customer story about Terminal X, a company that builds AI agent systems for investment firms. The headline result: Terminal X's grading agent, the "Eval Agent", running on GPT-6.1 Sol, agreed with a senior human reviewer 88.6% of the time across 500 scored judgments. The next-best model, which the article doesn't name, scored 86.4%. Note what "matches senior investor judgment" means here. A model-based grader's pass/fail verdicts on agent output were compared with a human's. No model was making investment decisions in this test.
The problem: "memory debt"
Terminal X's pitch starts with a problem it calls memory debt. An investment firm's judgment lives in a few senior people. When they leave, the firm keeps the documents but loses the reasoning behind them. The company wants to turn that reasoning into software the firm owns, running on-premise or in the firm's own cloud.
The hard part isn't writing a memo generator. It is (a) capturing standards nobody ever wrote down and (b) checking at scale that generated work actually meets them. The second part is where the Eval Agent comes in.
How the system is built
Terminal X uses 1: staff placed with the customer, in this case former finance professionals from JPMorgan and Blackstone. They spend weeks onsite reverse-engineering the memos the firm trusts. They also record unwritten processes, such as how the credit desk screens a borrower or which sources the co-investment team checks before writing a memo.
They encode this into hundreds of agents, customised by asset class and strategy, with sub-agents for each investment stage. The work is split between models by task:
GPT-6.1 Sol handles stages that need judgment and a large context window: synthesising the investment thesis, identifying key considerations, and assessing fit with the fund's 2, meaning the kinds of deals the fund is set up to do.
GPT-6 Luna handles high-volume steps where token cost matters more than depth.
The agents run inside each firm's 3, a secure space for sensitive deal documents. They read English, Korean, Japanese and Chinese. What they learn is stored as a firm 4: a versioned record of how one desk makes one kind of decision, including past analyses, investment committee outcomes, workflow instructions and 5 (the firm's official positions). Each Memory becomes a repeatable workflow. Every production query logs its steps, the sources it retrieved, the model used and its reasoning, so it can be audited later.
How the Eval Agent grades
The Eval Agent is an orchestrated workflow on GPT-6.1 Sol and other OpenAI models. It reads each output against a 54-criterion rubric. The questions fall into three groups:
Sourcing: are the numbers, 6 (comparable companies used for valuation), consensus figures and quotes pulled from the right sources?
Coherence: does the narrative hold together?
Alignment: does the recommendation tie back to the house view?
Every criterion is pass or fail. A fail carries a one-line reason and a better source or corrected text. That failure then becomes part of the Memory, as a new workflow instruction or a new test case for that desk. An illustrative example (mine, not the article's): if a memo values a target using the wrong peer group, the fail record names the correct peers, and future memos on that desk are checked for it. Analysts' manual corrections are folded in the same way. Ken Lee, Terminal X's CTO, describes the goal as the system coming to "think like their own team" over time.
The review pipeline is layered. The thesis is checked against the fund's mandate first, then against its recent deals. Uncertain verdicts are run again, and the agent reconciles the runs before reporting, a form of self-consistency checking. Terminal X says Sol's token efficiency on long inputs is what made this deeper pipeline affordable within the same budget. It also says the deeper review flagged deals that a single pass on a comparable model had let through.
The numbers
How much weight to give it
This is a vendor case study, and the evidence is thin in specific ways.
Sampling noise. At 500 samples, an 88.6% agreement rate has a standard error of about 1.4 points (my calculation, assuming independent judgments). Whether the 2.2-point gap is meaningful depends on whether both models graded the same 500 items, which would allow a paired comparison. The article doesn't say.
No human baseline. We aren't told how often two senior reviewers agree with each other. If humans agree with each other 90% of the time, 88.6% is near the ceiling. If they agree 95% of the time, there is real room left.
Unnamed competitor and missing denominators. The 25% drift figure has no base count. "25% more" could mean 5 extra catches or 50. The next-best model isn't identified, so readers can't check the result elsewhere.
One reviewer. The text refers to "the senior human reviewer", singular, so the benchmark may encode one person's preferences.
Still, the design itself is the useful part for practitioners. The pattern has four pieces: a fine-grained binary rubric, fails that carry corrections and become regression tests, re-running only uncertain verdicts, and routing between an expensive model and a cheap one by task. It also suggests that cheaper tokens on a strong model may matter mostly because they buy extra verification passes.
What's next
Terminal X plans to extend Memories from desks to individual investors, capturing each person's standard "as it is practiced, not only as it is described." The article doesn't explain how individual Memories would be kept consistent with the firm-level house view when the two conflict.
Remember
The 88.6% figure measures how often a grader agreed with one senior reviewer, not how well the investment calls themselves performed.
Every failed rubric check becomes a correction and a regression test, so the grader also trains the system over time.
Cheaper long-context tokens mattered mainly because they paid for extra checks: mandate, then recent deals, then re-runs of uncertain verdicts.
September 2024, NEJM AI. A prospective, multi-site validation found that sensitivity stayed high across different populations and that the model also flagged incidental lung and heart abnormalities.
6
9, open-weight multimodal models with 4B and 27B parameters. MedGemma goes beyond classification to reasoning: it combines radiographic findings with clinical history, answers clinicians' questions in natural language and runs on-device without internet. Google reports more than 10 million downloads.
Outdated systems. Brand's friends at small businesses found PHP versions so old that they no longer receive security patches at all.
Agreement with a senior reviewer: 88.6% vs 86.4% over 500 judgments, a 2.2-point (220 7) gap.
Style drift: about 25% more catches than the next-best model. Style drift is when a recommendation leaves the fund's mandate. The article's example is a buyout fund whose mandate is to find 8 candidates (listed companies to buy and delist) trading below 9 (net asset value). When scenario models suggest the market is turning, its deal agents start recommending distressed-credit positions or rescue financings. Those may be reasonable trades, but they belong to a different fund.
Cost: about 30% lower cost per query than the next-best model, at similar speed.
Adoption: 1,700 firm Memories, more than 1 million custom workflows, and weekly active users equal to 81% of monthly active users, a high stickiness ratio for enterprise software.
Savings: one private equity firm reported $2.4 million a year saved on due diligence and memo workflows. This is a self-reported figure.
