Today: what desk-scale GPU clusters can and can't do, cheaper ways to make model calls, and satellite embeddings that make later tasks small.
Picked from 23 new articles across 24 sources · arrow keys for other days
Today in 30 seconds
30s left
NVIDIA now supports eight-node DGX Spark clusters with 1 TB of memory. They run a 753B GLM model at 31 tokens/sec for one user. story 1 ↓
NVIDIA's Python GPU-loading tutorial warns that a read can succeed while quietly going through the CPU. Two separate settings decide which path it takes. story 2 ↓
OpenAI's new Decisions API returns probabilities for multiple-choice questions. It costs $0.10 per million input tokens, but OpenAI published no accuracy numbers. story 3 ↓
Microsoft's Flint-interactive compiles clicks and drags into data-level events, so agents describe chart interactions instead of writing rendering code. No benchmarks reported. story 4 ↓
Ai2's OlmoEarth v1.3 releases open embeddings for every 10-meter pixel. It beat AlphaEarth on 20 of 27 evaluations, and its final training run cost about $2,720. story 5 ↓
Anthropic gave its Agent Skills standard to the Agentic AI Foundation. Over 70 platforms, including ChatGPT, Cursor and VS Code, already support it.
1/5NVIDIA Technical Blog· Shara Tibken· Oct 22
NVIDIA supports eight-node DGX Spark clusters with 1 TB of memory
NVIDIA now officially supports linking eight DGX Spark desktop machines into one 1 TB cluster. Its benchmarks: the full 753B GLM 5.3 model at 31 tokens/second, decode up 4.4x, and near-linear training.
Why it matters · These are useful numbers on where desk-scale clusters help and where they don't: memory capacity and data-parallel training scale well, while single-stream tensor-parallel inference scales only partly.
Know this first
Context from the editor, not the article · 29 sec
1 · What a DGX Spark is
A small desktop machine from NVIDIA with 128 GB of memory shared by the CPU and GPU. A built-in ConnectX-7 network card lets several units be cabled together into a cluster.
2 · Tensor parallelism and all-reduce
To spread one model over several machines, each layer's weights are split across the GPUs. Every layer then needs an all-reduce, a step where all GPUs sum their partial results. During token-by-token decoding these exchanges are small and frequent, so latency dominates.
3 · Ring vs. switch
Nodes can be cabled in a ring, where each talks only to its neighbours, or through a network switch, which lets any node reach any other directly. Ring all-reduce passes data around in several hops but uses bandwidth efficiently.
2/5NVIDIA Technical Blog· Tanya Lenz· Oct 22
NVIDIA shows how to load files straight into GPU memory from Python
NVIDIA's tutorial uses the KvikIO Python library to read a 1 GiB file straight from an NVMe drive into GPU memory. It then uses runtime counters to prove the read actually skipped the CPU.
Why it matters · If you load data or checkpoints onto GPUs, your "direct" reads may be quietly going through the CPU. This shows how to check, and which settings decide.
Know this first
Context from the editor, not the article · 31 sec
1 · The data-loading bottleneck
Normally a file read goes from disk into CPU memory and is then copied to the GPU. That extra hop costs CPU time and memory bandwidth. It matters more as datasets grow and GPUs get faster.
2 · Direct memory access
With DMA (direct memory access), a device such as an NVMe drive writes into memory without the CPU copying each byte. Peer-to-peer DMA over PCIe lets the drive write straight into GPU memory, but only if the hardware topology and kernel settings allow it.
3 · Fallback paths
GPU I/O libraries usually fall back silently to the slower CPU path when the fast one isn't available. That keeps code portable, but it means a read that succeeds tells you nothing about how fast it was.
3/5OpenAI· Oct 22
OpenAI releases Decisions API for fast multiple-choice model calls
OpenAI's Decisions API is now generally available. It points its GPT-6 Luna model at questions with fixed answer sets and returns a probability, a choice or a score. OpenAI says it runs up to 10x faster than ordinary generation.
Why it matters · If you route, classify or gate agent steps with full LLM calls, this could cut your latency and cost. But OpenAI gives speed claims without accuracy numbers, so you'd need to test it yourself.
Know this first
Context from the editor, not the article · 31 sec
1 · Small decisions in LLM apps
Real LLM products make many tiny judgment calls: which category a ticket belongs to, which tool to call, whether to retry. Each is usually a full model call, so latency and cost grow with every step of an agent loop.
2 · Generation vs. constrained answers
A normal API call generates text one token at a time, and you parse the answer out of it. When only a few answers are allowed, you can instead read how likely the model thinks each option is. That is faster and gives calibrated-looking probabilities.
3 · Prompt caching
Many requests reuse the same long instructions or context. Providers can store the processed form of that shared prefix and charge much less to reuse it. That matters when you ask hundreds of similar questions.
4/5Microsoft Research· Chenglong Wang, Yuwei Xiao, Alper Sarikaya, Michel Galley, Chris Quirk, Jianfeng Gao· Oct 22
Microsoft's Flint-interactive lets AI agents describe chart interactions at a high level
Microsoft Research released Flint-interactive, a language for interactive charts. Agents map events like "user selected these countries" to updates like "highlight them", and a compiler does the rendering. The post claims better reliability but publishes no measurements.
Why it matters · It applies a familiar lesson about building agents: give the model a small, high-level language and let a compiler handle the fiddly parts, rather than having the model write fragile low-level code.
Know this first
Context from the editor, not the article · 32 sec
1 · Why interactive charts are hard
A static chart is a function from data to marks. Interaction adds state. Clicks have to be matched to the marks under the cursor (hit testing), mapped back to data rows, and then pushed out as updates to every linked view without breaking the layout.
2 · Declarative chart specs
Grammars like Vega-Lite describe a chart as data ("x = GDP, colour = continent") and leave the drawing to a renderer. Flint, the Microsoft base this work extends, works the same way: a chart spec goes into a compiler. The post doesn't detail Flint's grammar.
3 · MCP and MCP Apps
5/5Hugging Face· Oct 22
Ai2's OlmoEarth v1.3 releases open satellite embeddings for every 10-meter pixel
The Allen Institute for AI released OlmoEarth v1.3, an open model that turns a year of satellite imagery into 128 numbers per 10-meter pixel. It also published those numbers for the whole planet and a free browser tool for exploring them.
Why it matters · It's a clear case of "compute once, reuse everywhere": spend a lot on a foundation model's output once, and many later tasks shrink to small classifiers you can run on a laptop.
Know this first
Context from the editor, not the article · 28 sec
1 · Earth-observation satellites
Sentinel-1 (radar, sees through clouds), Sentinel-2 and Landsat (multispectral, capturing many color bands beyond what eyes see) photograph the whole planet every few days. That makes it hard to turn the pile of raw imagery into usable maps.
2 · What an embedding is
An embedding is a fixed-length list of numbers a model produces to summarize an input. Similar inputs get nearby vectors. A small classifier trained on those vectors can then do a new task without retraining the big model.
3 · Fine-tuning vs. embeddings
Fine-tuning means retraining a foundation model for each task, which needs GPUs and the raw imagery. Precomputed embeddings skip both: you download features someone already computed and fit a lightweight model on top.
Decode speedup with one to eight nodes: 4.43x with RoCEnante versus 3.05x without, at eight nodes.
NVIDIA now officially supports clusters of eight DGX Spark desktop machines. Each Spark has 128 GB of 1 (shared by CPU and GPU), so eight give 1 TB. Support in NVIDIA's Sync setup tool is due by the end of the year.
What 1 TB buys
Capacity is the main gain. The full GLM 5.3 from Z.ai, a 753-billion-parameter 2 model, is about 470 GB in 3 (NVIDIA's 4-bit number format). It wouldn't load on four nodes but runs on eight. With one user it decodes at 31 tokens/s. With eight concurrent users the total rises to 67 tokens/s, but each user gets only 18. The smaller GLM 5.3 Flash runs at about 80 tokens/s.
A faster 4 for decoding
Splitting a model across machines (5) means GPUs must sum partial results after every layer, a step called an all-reduce. The standard 6 library usually does this as a ring, passing data neighbour to neighbour over several rounds. That uses bandwidth well but adds latency, and latency is what matters for the tiny messages of token-by-token decoding.
NVIDIA's new 7 backend has every GPU send its input to all the others in one step, and then each GPU adds them up locally. That means fewer steps but more traffic, so it only wins with small messages and few nodes. It also needs a network switch. It is merged into the 8 kernel library and falls back to NCCL if its startup test fails.
On dense Qwen3.8-27B with one user, eight nodes decode at 52.6 tokens/s versus 11.9 on one: 4.43x, compared with 3.05x without RoCEnante. Prompt processing (9) scales much worse, only 1.94x on eight nodes.
NVIDIA's advice: a switchless ring is the cheapest good option at four nodes. A switch is needed to split the cluster into groups, to run any-to-any workloads, or to use RoCEnante.
Routing and training
10 uses a small classifier to send easy requests to MiniCPM5 2B and hard ones to Qwen3.8 Flash Next. On a tool-use benchmark the routed setup matched Qwen alone (91/100). Total throughput rose about 1.5x when half the requests went to Qwen and about 3x when a quarter did.
Training scales best, because each node keeps a full model copy and only syncs gradients once per step. Pre-training an 896.5M-parameter 11 model (Andrej Karpathy's minimal training stack) ran 7.79x faster on eight nodes, 97.4% efficiency.
Caveats
All of these are NVIDIA's own single-user benchmarks, with no comparison against other hardware. The memory is the real gain. Inference speed still scales well short of linearly.
NVIDIA announced official support for eight-node DGX Spark clusters at GTC Berlin. DGX Spark is its desktop AI machine with 128 GB of 1, meaning the CPU and GPU share one pool. Eight units give 1 TB. The NVIDIA Sync tool's Cluster Assistant, which handles setup, will support eight-node clusters by the end of the year. The accompanying blog post is a vendor benchmark write-up covering four things: serving very large models, a faster communication backend, scaling across network topologies, and two workloads that split up well (routing between models, and pre-training).
Memory first: the 753B model
The main gain is capacity. Four nodes (512 GB) could not load the full GLM 5.3 from Z.ai, a 753-billion-parameter 2 model (one that activates only a subset of "expert" sub-networks per token). In 3, NVIDIA's 4-bit floating-point format, the checkpoint is about 470 GB, so it fits across eight nodes. The model was measured with 4, meaning multi-token prediction drafting three tokens ahead as a form of speculative decoding, and an 8K-token input per request:
Concurrent users
Prefill total (tok/s)
Decode total (tok/s)
Decode per user (tok/s)
1
1,600
31
31
4
1,600
51
21
8
1,600
67
18
Prefill, the processing of the prompt, stays flat at about 1,600 tokens/s in total. Decode throughput rises with more users while each user's speed falls. 31 tokens/s is usable for interactive work, though not fast. The smaller GLM 5.3 Flash variant reached about 80 tokens/s on the same cluster.
A latency-tuned 5: 6
Tensor-parallel inference splits each layer across GPUs and then runs an all-reduce to sum the partial results. The standard library is 7 (NVIDIA Collective Communications Library), which typically uses ring all-reduce: data passes between neighbours over several rounds. That is bandwidth-efficient, but each round adds latency. For decoding one token at a time across a few nodes, the messages are tiny, so latency dominates.
RoCEnante takes the opposite approach. The name refers to RoCE, RDMA over Converged Ethernet, which lets one machine write directly into another's memory over Ethernet. Each rank (GPU) sends its full input to every other rank in one all-to-all exchange, and then every rank sums all the inputs locally. A single GPU kernel coordinates both steps. A rough analogy: instead of passing a running total around a table one person at a time, everyone shouts their number to the room at once and each person adds it up. That takes one step instead of many, but there is much more total traffic. As message size or node count grows, the extra traffic costs more and the ring wins again. NVIDIA says NCCL stays the default and is better for high concurrency, larger clusters or ring topologies.
The benchmark used dense Qwen3.8-27B in NVFP4 with speculative decoding off. At eight nodes, decode speedup was 4.43x with RoCEnante versus 3.05x without, which NVIDIA describes as a 45% improvement in scaling efficiency. The post also reports about 5% higher 8 and 21% higher decode throughput at four nodes, and about 18% higher decode at eight nodes on the non-quantized model. The backend is merged into 9, an open-source library of inference kernels, and falls back to NCCL if its startup self-test fails. It needs a switch, because every node must reach every other node directly. The original one-shot all-reduce idea came from a community contributor (Jason Cook/el8).
Scaling by topology
Same Qwen 27B model, 8,192 input tokens, 256 output tokens, one user:
Nodes
Topology
Prefill tok/s (scaling)
Decode tok/s (scaling)
1
—
2,635 (1.00x)
11.86 (1.00x)
2
Direct cable
3,285 (1.25x)
21.71 (1.83x)
4
Ring
4,446 (1.69x)
32.82 (2.77x)
4
Switch + RoCEnante
4,556 (1.73x)
36.34 (3.06x)
8
Ring
4,962 (1.88x)
40.11 (3.38x)
8
Switch + RoCEnante
5,105 (1.94x)
52.58 (4.43x)
Decode scales reasonably, about 55% efficiency at eight nodes in the best case. Prefill scales poorly, under 2x on eight nodes. The post doesn't discuss why. A plausible reading is that prefill moves large activation tensors, so it is limited by network bandwidth.
On topology, NVIDIA recommends a switchless ring for the simplest, cheapest four-node inference box, because NCCL's ring algorithms match the cabling. A switch is needed to split the cluster into independent groups (for example two groups of four), to run any-to-any workloads such as MPI-based HPC jobs, or to use RoCEnante.
Routing between models: 10
NeMo Switchyard puts a lightweight classifier in front of model pools, with a load balancer across replicas, so easy requests go to a small model. In NVIDIA's prototype, OpenBMB's MiniCPM5 2B acted as both the classifier and the model for simple tasks, and Qwen3.8 Flash Next handled hard ones. The eight nodes held three Qwen replicas (two nodes each) and two MiniCPM replicas (one node each). On a 92-case tool-use benchmark the routed setup scored 91/100, the same as Qwen alone. The post does not explain how 92 cases map to a score out of 100. MiniCPM serves roughly ten times more requests per second per replica, so aggregate throughput rose about 1.5x when half the traffic went to Qwen and about 3.0x when a quarter did. The routing policy is a YAML file, and NVIDIA ships a sample.
Training: near-linear
Training here is data-parallel: every node holds a full copy of the model and syncs gradients once per step. That means far less communication than tensor-parallel decoding. NVIDIA pre-trained a 11 model from scratch. Nanochat is Andrej Karpathy's minimal open-source training stack for a chat model. The model had about 896.5M parameters, 20 layers, a hidden dimension of 1,280 and 2,048-token sequences, trained with a 524,288-token global batch:
Nodes
Tokens/s
Speedup
Efficiency
1
8,741
1.00x
100%
2
17,202
1.97x
98.4%
4
33,640
3.85x
96.2%
8
68,079
7.79x
97.4%
The code is on GitHub.
What to take from it
The 1 TB of memory is the real change. It is what makes a ~470 GB model runnable at a desk at all. Inference speed scales well short of linearly, and prefill hardly scales, so the cluster suits a single developer or a few concurrent users, not production serving. Every number comes from NVIDIA with one user and fixed sequence lengths, and the post gives no comparison against other hardware.
Remember
Prefill on dense Qwen 27B barely scales: 1.94x on eight nodes, against 4.43x for decode.
One-shot all-to-all all-reduce beats ring for tiny messages at few nodes but needs a switch.
Data-parallel training scaled at 97.4% efficiency, far better than tensor-parallel inference on the same hardware.
The two decision points: KvikIO picks cuFile or its POSIX path, then cuFile picks the direct or CPU-staged path.
NVIDIA has published a tutorial on reading files straight into GPU memory from Python. It uses 1, a Python and C++ file-I/O library, on top of 2, NVIDIA's technology for moving data from storage directly into GPU memory by 3 (direct memory access) without staging it in CPU memory. The post has no speed benchmarks. Its value is in explaining who decides whether a read really skips the CPU, and how to check.
Why bother
The usual path reads a file into host memory and then copies it to the GPU. GDS removes that hop and cuts CPU load. That matters because, per the 2025 Stanford AI Index, AI training datasets double roughly every eight months. But GDS's main interface, 4, is a C API. KvikIO gives it a Python interface.
Two layers, two decisions
The key point is that a read passes two separate path selectors:
KvikIO decides whether to call cuFile. In its default AUTO mode it tries cuFile and falls back to its own CPU-staged 5 path if cuFile is unavailable. Reads under 16 KiB always take the POSIX path.
cuFile then decides whether to use the accelerated path, through the nvidia-fs kernel module, or its own CPU-staged compatibility path.
So a successful read, or even reaching cuFile, doesn't prove the fast path was used.
Checking it
On an H100 with a local NVMe drive, the script is a few lines. It allocates a 1 GiB 6 array on the GPU and calls kvikio.CuFile(path, "r").read(buffer). Run it with CUFILE_STATS_LEVEL=3 and inspect it with the gds_stats tool. KvikIO's default 4 MiB task size splits the read into 256 operations. The stats showed nvfs=256, posix=0, meaning all of them took the direct path. Forcing cuFile's compatibility mode moved the counts to posix. Setting KvikIO's own compatibility mode on bypassed cuFile entirely, so cuFile recorded nothing.
Tuning has two layers too
KvikIO splits a read into tasks, and cuFile can split each task again using settings in cufile.json. With 4 worker threads, a 1 GiB task becomes four 256 MiB pieces. The CuPy buffer was never registered with cuFile, so data actually lands in a registered GPU 7, in slabs of at most 16 MiB, and is then copied over.
The setup was demanding. The drive needed at most one 8 bridge to the GPU, 9 and ACS (PCIe isolation features) had to be disabled, and the nvidia-fs module had to be loaded. The post doesn't measure how much faster the direct path is. Its practical advice is to check gds_stats before assuming you're getting GDS.
NVIDIA has published a hands-on tutorial on reading files directly into GPU memory from Python. It uses 1, a Python and C++ file-I/O library, on top of NVIDIA 2. There are no throughput benchmarks. What the post offers instead is something more useful: a precise account of who decides whether a read actually bypasses the CPU, and how to verify it.
The problem
NVIDIA motivates the work with the Stanford AI Index 2025 figures. Compute for notable AI models doubles roughly every five months, and LLM training datasets roughly every eight. Feeding accelerators from storage is becoming a bottleneck in its own right. The conventional path reads a file into CPU (host) memory and then copies it to the GPU. GDS removes that staging step. Supported storage moves data by 3 (direct memory access, where the device writes into memory without the CPU copying bytes) straight into GPU memory. This also cuts CPU utilization.
The catch is the interface. GDS's main programming interface is 4, a C API, while nearly half of new AI projects on GitHub are primarily Python. KvikIO wraps cuFile in idiomatic Python and C++ and adds its own controls.
Two decision points, not one
The post's central point is that a KvikIO read passes through two independent path selectors (see the diagram).
So reaching cuFile proves only that KvikIO chose cuFile. It does not prove the accelerated path ran. The read's return value (bytes read) is no help either. It confirms the full file arrived, not how it got there.
The setup
The test machine was an x86_64 box with an H100 and CUDA 13.3. It had cuFile 1.18 and a local NVMe drive formatted ext4. The GPU and drive had a "PIX" topology, meaning at most one 6 bridge between them. 7 and ACS were disabled so PCIe peer-to-peer transfers aren't blocked. IOMMU is the memory-remapping unit for devices; ACS is a PCIe access-control feature that can force peer traffic through the CPU's root complex. The nvidia-fs module was loaded. These prerequisites are where real deployments often fall down.
The code itself is short. pip install kvikio-cu13 pulls in 8. The script allocates a 1 GiB CuPy array, opens the file with kvikio.CuFile(path, "r"), and calls file.read(gpu_buffer).
Verifying the path
To see what happened, run with CUFILE_STATS_LEVEL=3 and keep the process alive with python3 -i. Then inspect it with NVIDIA's gds_stats tool. The per-GPU counters split operations into nvfs (accelerated, via 9) and posix (cuFile's compatibility path).
With KvikIO's default 4 MiB task size, the 1 GiB read becomes 256 operations. The stats showed n=256, nvfs=256, posix=0, so every operation took the direct path. Three further runs isolate each layer:
KVIKIO_COMPAT_MODE=off: KvikIO must call cuFile, and counts appear under nvfs.
Add CUFILE_FORCE_COMPAT_MODE=true: KvikIO still calls cuFile, but operations now appear under posix.
KVIKIO_COMPAT_MODE=on: KvikIO bypasses cuFile entirely, and cuFile records nothing.
How a big read gets split, worked through
Both layers can chop up a request. KvikIO first splits by KVIKIO_TASK_SIZE, schedules the pieces with KVIKIO_NTHREADS, and sends each task as a separate cuFile request. cuFile may then split each request again. Its cufile.json settings control this: parallel_io, max_io_threads, max_request_parallelism, min_io_threshold_size_kb and max_io_queue_depth.
The tutorial configuration had 4 cuFile worker threads, a request-parallelism cap of 4, and an 8 MiB minimum piece size.
Default 4 MiB tasks: 256 cuFile requests, each handled as a single piece. This was the nvfs=256 run.
Task size set to 1 GiB: one request reaches cuFile. The CuPy buffer was never registered with cuFile, so cuFile uses its unregistered-buffer scheduling. It splits into min(4 threads, 4 parallelism) = 4 pieces of 256 MiB, well above the 8 MiB floor. Three pieces go to worker threads and the calling thread handles the fourth.
Context and what's left open
KvikIO is part of NVIDIA's CUDA-X data-science libraries, formerly RAPIDS. It grew out of the GDS integration in cuDF, NVIDIA's GPU DataFrame library, where parallel cuFile reads sped up loading of Parquet and ORC (columnar file formats). It has since been used in cuCIM pathology workflows and cuPhoton's xDataReader. NVIDIA also cites GPU-accelerated Presto work.
The post gives no throughput or CPU-utilization numbers comparing the paths. It also doesn't say which task size or cuFile settings perform best. The bounce-buffer detail suggests registering the destination buffer could avoid an extra on-GPU copy, but the tutorial doesn't measure that. It also omits failure and unsupported-configuration paths. The practical lesson is a verification habit: check gds_stats for nvfs rather than posix before assuming your pipeline is using GDS.
Remember
KvikIO's default AUTO mode silently falls back to CPU-staged reads, so a successful read proves nothing about speed.
Run with CUFILE_STATS_LEVEL=3 and check that gds_stats counts reads under nvfs, not posix.
Unregistered GPU buffers go through bounce buffers and an extra copy, even on the accelerated path.
OpenAI has made its Decisions API generally available. It is a dedicated endpoint for questions with a fixed set of possible answers, running on 1, an OpenAI model the announcement doesn't otherwise describe. OpenAI says it makes decisions "up to 10x faster" than the same model called through the 2, the company's standard text-generation endpoint.
The problem
Applications and agents make many small calls: classify this input, route this request, choose the next step. Today each one is usually a full generative call. You prompt for a label, wait for decoding, pay for output tokens and parse the result. Across an agent loop, that latency and cost add up.
How it works
You define a question and its allowed answers, then pass context as text, images or 3. One request can carry up to 400 questions, double the beta limit. Each question returns one of three result types:
predicate: a 0–1 probability that a condition is true ("is this product damaged?").
choice: one of your options (continue, retry, clarify), plus per-option probabilities and a confidence value.
score: a rubric rating computed as the probability-weighted average of level indices. With illustrative numbers: levels 0/1/2 with probabilities 0.2/0.5/0.3 give a score of 1.1.
OpenAI doesn't describe the internals. Every output is a probability, and output tokens aren't billed. That suggests, though the article doesn't say so, that the system reads the model's likelihoods over the allowed answers rather than generating text.
What's new since beta
Native integration with 4, OpenAI's real-time voice product. A configured trigger fires a decision.event as the transcript updates. OpenAI's demo uses it to pick an avatar's facial expression; a support bot could use it to decide whether to escalate.
5 for reused context.
US and EU 6, plus more capacity in Asia.
Pricing
Input costs $0.10 per million tokens, or $0.01 per million when cached. Output is free. 7, OpenAI's option not to store request data, is available to eligible customers.
What's missing
The examples are mostly OpenAI's internal teams: categorizing finance transactions, and scoring press coverage by theme. None come with numbers. The "10x" figure has no stated baseline or latency distribution. There are no accuracy results, and nothing on whether the probabilities are 8, meaning whether a 0.8 is right about 80% of the time. That matters, because the whole point is to set thresholds on those probabilities. Before switching, test it on one high-volume classification call in your own system.
OpenAI has made its Decisions API generally available. It is a dedicated endpoint for questions whose answers come from a fixed set, and it runs on 1, an OpenAI model the announcement doesn't describe further. OpenAI says it returns decisions "up to 10x faster" than calling the same model through the 2, OpenAI's general-purpose text-generation endpoint. Billing covers input tokens only.
The problem it targets
OpenAI frames this around the many small calls inside applications and agents: classify this input, route this request, choose the next action. Each one is simple. But when every step of an agent loop needs a full generative call, the latency and cost add up. The usual workaround is to prompt a general model to output a label and then parse it. That works, but you pay for generated tokens, wait for decoding, and get no reliable sense of how confident the model is.
How it works
You define a question and its allowed answers, then supply context as text, images or 3 (structured data passed directly rather than pasted in as text). One request can carry up to 400 questions, double the limit during the public beta. Each question returns one of three result types:
predicate: a probability from 0 to 1 that a condition holds, such as "does this product have visible damage?"
choice: one of your predefined values, such as continue, retry or ask for clarification. The response also includes per-option probabilities and a confidence value.
score: a rating against an ordered rubric, computed as the probability-weighted average of the level indices.
A worked example of the score type, with illustrative numbers: take a damage-severity rubric with levels 0 (none), 1 (minor) and 2 (severe). If the model assigns probabilities 0.2, 0.5 and 0.3, the score is 0×0.2 + 1×0.5 + 2×0.3 = 1.1. You get a continuous value rather than a hard label, which helps with thresholds and ranking.
OpenAI doesn't explain what happens inside. Because every result type is a probability or derived from one, and output tokens are free, the system most likely reads the model's likelihoods over the allowed options instead of generating free text. That inference is ours, not something the article states. It would explain both the speed and the pricing.
What changed since beta
A native integration with 4, OpenAI's real-time voice product. You add a decision configuration to session.triggers in a Live session. As transcript updates arrive, Live emits a decision.event that your application acts on. OpenAI's demo picks an avatar's facial expression (a wink, a smile) from what the user says. A support agent could use the same mechanism to decide whether to clarify, route or escalate.
5 for reused context.
Pricing
Input costs $0.10 per million tokens.
Cached input costs $0.01 per million tokens.
Output tokens are free.
Cached writes come in 30-minute and 12-hour tiers. Their rates are in OpenAI's pricing guide, not the announcement.
6, OpenAI's option not to store request data, is available to eligible customers.
Since a 400-question batch over shared context is mostly input, caching probably dominates the real cost of heavy use.
Usage examples
OpenAI's examples are internal. A team behind a product called "dots" uses it to cut response times. The Personal Finance team uses it to categorize transaction descriptions. The Communications team classifies press coverage by theme and scores whether each piece reinforces or challenges key narratives. Customers reportedly use it for attributing AI costs and for interactive language learning. None of these come with numbers.
What the announcement leaves out
The evidence is thin. "Up to 10x faster" comes with no baseline prompt, no context length and no latency percentiles. There are no accuracy figures, and no comparison against Luna through the Responses API or against a small fine-tuned classifier. The article never says whether the returned probabilities are 7, meaning whether a 0.8 is right about 80% of the time. That matters most here, because the API's main selling point is giving you probabilities to set thresholds on. It also doesn't say how the 400 questions share context, or whether answers to related questions are consistent with each other.
For a practitioner, the case to test is concrete: replace one high-volume routing or classification call and compare latency, cost and agreement with your current labels. OpenAI's own advice is to start with one recurring decision, using its Decisions Playground and developer guide.
Remember
Outputs are probabilities, choices with per-option probabilities, or probability-weighted rubric scores. None are free text.
Billing is input-only: $0.10 per million tokens, $0.01 when cached. Batching 400 questions over shared context favors caching.
OpenAI gives no accuracy or calibration data and no baseline for the '10x faster' claim.
The Model Context Protocol (MCP) is a standard way for AI assistants to call outside tools. MCP Apps extends it so a tool can return interactive UI that renders inside the chat window.
The design: agents map semantic events to canvas updates while the compiler handles rendering and runtime.1 / 4
Microsoft Research has released Flint-interactive, a language that lets AI agents build interactive charts without writing low-level rendering code. It extends 1, the team's existing system for specifying charts, which turns a chart description into a rendered image through a compiler.
The problem
Agents already make decent static charts. Interactive ones are harder to get right: zooming, 2 (dragging a selection box over) one chart to highlight the same items in another, or clicking a point to highlight a related sentence. The authors name two obstacles even with frontier models. The first is cost: an agent rarely succeeds on the first try, and each round of debugging uses tokens. The second is reliability: interactive code tends to break layouts or interactions, and fixes cause new bugs. They argue that the interaction features built into charting libraries aren't expressive enough for custom charts.
How it works
The design splits the work in two.
The compiler owns visual logic. It turns raw clicks, drags and key presses into , meaning "the user selected these countries" rather than "a click happened at pixel (312, 88)". It also does the (working out which mark is under the cursor) and all the redrawing.
For example, brushing countries on a 1952 GDP-versus-life-expectancy scatterplot creates a selection event. A preset called linked-brush then highlights the same countries on the 2007 plot. Events can also go to tables, text or app state, and outside widgets can send operators back. That is how chart-and-table dashboards fit together.
Ways to use it
In chat, via Flint-MCP. 6, the Model Context Protocol, is the standard way for assistants to call tools, and 7 lets those tools render interactive UI in the chat. You ask for a chart and its interactions, and what you select can feed your next question.
The caveat
This is an announcement, not a paper. Cheaper, more reliable generation is the main reason given for the design, and the post measures neither: it has no success rates, token counts or comparison with agents writing D3 or Vega-Lite. The approach is plausible, since a narrow language plus a deterministic compiler usually makes LLM output more reliable. The cost is that agents can express only what Flint's vocabulary covers.
What shipped
Microsoft Research has released Flint-interactive, a language for adding interactivity to charts built with 1, the team's existing chart-specification system with a compiler. The team is led by Chenglong Wang and includes Jianfeng Gao. The intended authors are AI agents, though people can use it too. An agent writes a small interaction_spec next to the usual chart spec, and the compiler handles everything between a mouse event and a redrawn pixel.
The problem the authors describe
Agents are good at making static charts. Users often want more: zooming in, 2 (dragging a selection box over) one chart to highlight the same items in another, or clicking a point to highlight a related sentence in a report. The authors want the quality of interactive pieces in data journalism, and they name two obstacles even with frontier models:
Generation cost. An agent rarely gets an interactive visualization right on the first try. Rounds of reasoning, coding and debugging use up tokens and time.
Reliability. Interactive code tends to produce bad layouts or broken interactions, and fixing one problem often causes another.
Interaction features built into existing charting libraries don't solve this, they argue, because those features aren't expressive enough for custom charts. Their diagnosis is that there is no language for interactions that is both expressive and high-level.
How it works
Flint Canvas, the runtime, is a state machine. It tracks what the user is doing and what the chart currently looks like. The design splits the work into two layers.
Visual logic, owned by the compiler. Raw triggers (clicks, drags, key presses) become 3: descriptions of which chart elements and which data rows the user touched. Going the other way, the compiler turns 4 into actual canvas changes. It handles 5, which means working out which mark is under the cursor, and all the low-level redrawing.
Interaction logic, written by the agent or user. The author only maps semantic events to update operators. The post names three operators:
set-style, for example to highlight data
set-annotation, to label a point
set-viewport, to pan or zoom
Semantic events can also flow out of the chart to update tables, widgets or application state. Outside code can send update operators back in.
Here is how one interaction plays out. Take the post's Gapminder example: two scatterplots of countries in 1952 and 2007, with GDP per capita on the x-axis, life expectancy on the y-axis, and colour by continent. The user drags a box over a cluster in the 1952 plot. The compiler turns that drag into an event meaning "these countries were selected". A preset called linked-brush maps that event to set-style on the same countries in both plots. The agent never sees pixel coordinates or renderer-specific mark objects. It works only at the level of "these data were selected, so highlight them."
The authors argue this separation works because events and updates compose. The same event can drive a chart, a table and a paragraph at once, which is how dashboards and data stories get built.
Three ways to use it
There is also an "interaction-author" skill for coding agents, and an invitation to propose new presets on GitHub.
What the evidence does and doesn't show
This is a release announcement, not a paper. The claims of lower generation cost and higher reliability are the main reasons given for the design, but the post includes no measurements. It gives no success rates, no token counts, and no comparison against agents writing D3 or Vega-Lite directly. The examples are demos, not evaluations.
The approach is still plausible for a well-known reason. Shrinking what the model has to write, and giving the hard, stateful parts to a deterministic compiler, generally makes LLM output more reliable. That is why constrained domain-specific languages beat free-form code in many agent tools. The trade-off is just as familiar: the agent can express only what Flint's vocabulary allows. Anything beyond the presets needs the programming API, which brings back custom code, though at the level of data events rather than marks.
Open questions the post leaves:
How large is the preset vocabulary?
How does Flint's expressiveness compare with Vega-Lite's selection-parameter system?
How does it perform on large datasets?
How often does an agent actually get an interaction_spec right on the first try?
Remember
The core design hands hit testing and rendering to a compiler; agents only map data-level events to a few update operators.
The same events can drive charts, tables, text and app state, which is how linked dashboards are built.
The post claims lower token cost and higher reliability but gives no evaluation or baseline.
The pipeline: satellite time series pass through a vision transformer, an attention pooler and a linear layer, producing 128 numbers.1 / 3
What's new
The Allen Institute for AI (Ai2) released OlmoEarth v1.3, a satellite foundation model. With it came yearly embeddings precomputed for the whole planet at 10-meter resolution, and a free browser tool called the OlmoEarth Embeddings Explorer. The weights, pretraining data and code are all open.
Until now, using OlmoEarth meant fine-tuning it for each task in OlmoEarth Studio, Ai2's hosted platform. That required GPUs and enough storage to download satellite imagery. Precomputed embeddings remove both requirements.
How it works
For each location, the model reads a year of time-series data. That includes radar from 1 and multispectral imagery from 2 and Landsat, which capture many light bands beyond visible color. The data goes through three stages:
a 3 (a transformer-based image model)
an 4, a learned layer that merges outputs into one vector per patch of land
a linear layer that shrinks that vector to 128 numbers per 10-meter pixel
The first 64 numbers also work as a smaller embedding on their own.
Once those vectors exist, a crop map or land-cover map becomes a small job. You label a few points, train a simple classifier on their vectors, and run it locally.
The numbers
Ai2 compared v1.3 with 5 (Google DeepMind's satellite embeddings) and Tessera v2 on 27 evaluations across six continents. The 128-dimensional version scored best on 20 of them. The 64-dimensional version scored best on 18. The tests include 6, a new tropical crop-type benchmark from Ai2. The post gives win counts, not margins.
Training was cheap. v1.3 used 53% of v1.2's compute and 18% of v1's. Its final pretraining run cost about $2,720 in rented H100 GPU time, compared with $5,100 for v1.2 and $15,060 for v1. It still matches v1.2 when used as a base for fine-tuning.
Change detection without training
In Peru's Madre de Dios region, illegal gold mining cleared more than 1,200 hectares near the Tambopata reserve before a 2019 crackdown. To track it, you compare each pixel's vector across years, with no training at all.
The 2017–2020 comparison shows mining spreading southeast. The 2020–2025 comparison shows that area quiet, while the older zone to the west keeps changing. Some of that change is regrowth and some is renewed digging. The embeddings can't tell those apart on their own.
The Explorer and caveats
In the Explorer you can switch between years, find places that look similar, train a classifier from a few clicks, and export the result as a 7, a standard map image format. No account is needed.
The archive holds about 150 TB per year. Coverage is global for 2023–2025 and limited to select countries for 2017–2022. For each 20-km tile, provenance tables list the exact source satellite scenes. Because the embeddings are annual, changes within a year get averaged out.
What shipped
The Allen Institute for AI (Ai2) released OlmoEarth v1.3, a satellite foundation model, together with three things built from it. The first is a precomputed set of yearly embeddings covering the whole planet at 10-meter resolution. The second is a browser tool called the OlmoEarth Embeddings Explorer. The third is a technical report. The model weights, pretraining data and pretraining code are all open, under the same OlmoEarth Artifact License used for v1 and v1.2.
Until now, OlmoEarth was mainly used by fine-tuning it in OlmoEarth Studio, Ai2's hosted platform. You trained a model for each task, then ran it over satellite imagery to produce a map. That needs GPU access plus the ability to download and store large amounts of imagery. Few conservation groups, local governments or researchers have both.
The mechanism: one vector per pixel per year
For each location, v1.3 reads a full year of time-series data from three sources. 1 is radar, 2 is multispectral optical imagery, and Landsat is the long-running US optical program. From that year of data it produces one 128-dimensional vector for every 10-meter pixel on Earth.
The pipeline has three stages, according to Ai2's diagram:
A 3 (an image model built on the transformer architecture) processes the stacked satellite time series.
An 4, a learned layer that weights some inputs more than others when merging them, compresses the transformer's output into one vector per patch of land.
A linear projection reduces that vector to 128 numbers.
The first 64 dimensions also work as an embedding on their own. Ai2 doesn't say how this was trained. The property resembles "Matryoshka" embeddings, where training puts the most useful information in the leading dimensions so you can truncate the vector and lose little. That comparison is mine, not the article's. In practice, you can use half the storage and give up only a little accuracy.
With embeddings precomputed, many mapping tasks become small modeling problems. An illustrative example: to map one crop in a district, label a few dozen pixels as "crop" or "not crop". Then train a simple classifier on their 128-number vectors and apply it to every pixel in the region. All of this runs locally with no GPU, because the expensive work was done once when Ai2 computed the embeddings.
Results
Ai2 compared v1.3 against two other embedding products: 5 (AEF), Google DeepMind's satellite embedding model, and Tessera v2. The comparison covered 27 evaluations across six continents and a range of tasks.
128-dimensional embeddings: best on 20 of 27 evaluations, and best overall.
64-dimensional embeddings: still best on 18 of 27.
Generalization: Ai2 says its embeddings hold up best when a classifier runs on places it has never seen. This was tested partly with 6, a new crop-type benchmark for tropical farming that Ai2 introduced with this release.
The blog post reports win counts, not how large the margins are. It also doesn't say which seven evaluations v1.3 loses. Readers who care about a specific task should check the technical report.
As a fine-tuning base, v1.3 matches v1.2 on research and partner datasets. So the work to make good embeddings did not cost it anything on that front.
Example uses Ai2 lists include classifying smallholder crops in Kenya, detecting brick kilns west of Dhaka, assessing fire damage in Indonesia, and tracking gold mining in the Peruvian Amazon.
Training got much cheaper
v1.3 is the cheapest OlmoEarth model so far. It used 53% of v1.2's compute time and 18% of v1's. Its carbon emissions were 50% and 22% of those models' respectively. In rented H100 GPU time for the final pretraining run of each Base model:
Model
Final-run cost
v1 Base
$15,060
v1.2 Base
$5,100
v1.3 Base
$2,720
These figures cover the final run only. They exclude experiments, failed runs, and the much larger cost of computing embeddings for the whole planet. Even so, about $2,720 puts reproducing the model within reach of an academic lab.
A worked case: gold mining in Madre de Dios
Ai2's main demonstration needs no training at all. You compare each pixel's embedding across years, and pixels whose vectors moved a lot are places where the land changed. (The post doesn't say which distance measure it uses.)
Around La Pampa in Peru, illegal gold mining peaked in 2017–2018. According to Amazon Conservation's MAAP monitoring program, it cleared more than 1,200 hectares of forest in the buffer zone of the Tambopata National Reserve. In February 2019, Peru's Operation Mercurio shut most of it down.
2017 to 2020: the change map lights up where mining was spreading southeast.
2020 to 2025: that southeastern corner has gone quiet. Change now sits in the older mining zone to the west, which mixes forest regrowth with renewed digging.
This matches MAAP's ground reports that pockets in the northwest kept operating. The Associated Press reported in August 2026 that Peru sent navy patrols back into the reserve after prosecutors found mining inside Tambopata itself. One caveat: the change map shows that a pixel changed, not why. Regrowth and new digging both show up as change, so you still need imagery or local knowledge to tell them apart.
The Explorer and the archive
The Embeddings Explorer shows the embeddings as a false-color map of the planet. You can switch between years and pull up the underlying Sentinel-2 imagery to check what you see. You can click a place to find others that look similar, or drop a few labeled points, train a small classifier in the browser, and export the result as a 7 (a standard georeferenced image file). It needs no account and no setup. Ai2 points out that a first test that used to take an hours-long fine-tuning run now takes seconds. That can help decide whether a task deserves fine-tuning at all.
The archive is hosted on 8, a public data-sharing platform. It holds about 150 TB per year and covers land plus nearby coastal waters. At launch, 2023 through 2025 are global, while 2017 through 2022 cover only select countries. For every 20-kilometer tile and year, open provenance tables list the exact Sentinel-1, Sentinel-2 and Landsat scenes the model read, with links to the public catalogs that host them, so you can trace any pixel back to its source imagery.
Limits and open questions
The embeddings are annual, so anything that happens within a year, like flooding or a growing season, gets averaged into one vector. For custom date ranges, sensor combinations or resolutions, Ai2 points users to OlmoEarth Studio or a tutorial for generating embeddings on their own hardware. Coverage before 2023 is partial. The headline benchmark result is a win count from Ai2's own evaluation suite, which includes a benchmark Ai2 created. Independent comparisons on other tasks would make the claim stronger.
Remember
The first 64 of 128 dimensions work as a standalone embedding and still lead on 18 of 27 evaluations.
Final-run pretraining cost fell from $15,060 (v1) to $2,720 (v1.3), with fine-tuning quality equal to v1.2.
Comparing embeddings across years shows where land changed, but not why. Regrowth and new mining look alike.
KvikIO decides whether to call cuFile at all. Its compat_mode setting can be AUTO, ON or OFF. AUTO tries cuFile and falls back to KvikIO's own 5 backend, which stages data through host memory, if cuFile is unavailable. Reads smaller than gds_threshold (16 KiB by default) skip cuFile and take the POSIX path directly.
cuFile decides how to move the bytes. Depending on system support, file system and its cufile.json configuration, it uses either the accelerated path through the nvidia-fs kernel module or its own CPU-staged compatibility path.
Those 256 MiB pieces are scheduling units, not transfers. Because the destination isn't registered for DMA, cuFile reads into a registered GPU "10" and copies into the destination. Since GDS 1.17 these buffers can be configured as slabs, capped here at 16 MiB. So each 256 MiB piece actually moves in smaller chunks, and the 8 MiB setting does not set transfer size.
Where 8 MiB does matter: a 16 MiB request would split four ways into 4 MiB pieces, below the floor. cuFile instead makes two 8 MiB pieces. KvikIO's task size must be at least 16 MiB for cuFile ever to see such a request.
Inference residency in the US and EU, meaning processing stays in that region, plus added capacity in several Asian regions.
JSON inputs and images inside question instructions.
3
4
The agent writes interaction logic. It maps events to a few 5: set-style to highlight, set-annotation to label, set-viewport to pan or zoom.
In apps, through Flint Interactive Canvas, which emits events your code handles.
Through a programming API for custom interactions. One demo recreates 8, a research technique where you drag a point along its path through time and the whole chart moves to that moment.
In chat. Through Flint-MCP (an 6 server; MCP is the Model Context Protocol, the standard way assistants call tools) and 7 (which let tools render interactive UI inside a chat), you can ask an agent for a chart and the interactions you want. The chart renders in the conversation. What you select can be passed back as context for your next question.
In an app. Flint Interactive Canvas emits semantic events your code can handle, for example to filter a table or update a second chart. Your widgets can send operators back. One example links a country table and a bubble chart: clicking the United States row (GDP per person $42,952, life expectancy 78.2) highlights its bubble, and clicking the bubble highlights the row.
Custom interactions. A programming API lets you define a trigger plus a handler that turns events into a sequence of updates. For example, a handler can highlight a clicked country and label it by name, combining set-style and set-annotation. The most ambitious demo recreates 8, a published research technique where you drag a point along its path through time and the whole chart moves to that moment. Here it shows the seasonal temperature and precipitation cycles of five US cities.
