Today's stories are about changing core parts of live systems: storage defaults, transport protocols, borrowed hardware and user code, plus agents that now design their own teams.
Picked from 38 new articles across 23 sources · arrow keys for other days
Today in 30 seconds
30s left
Postgres 19 switches its default compression for large values to LZ4. Values under 2,032 bytes still aren't compressed at all. story 1 ↓
Instagram Live replaced RUSH, Meta's own upload protocol for nearly ten years, with the open Media over QUIC standard and kept the same quality. story 2 ↓
Meta fills idle inference hardware with optional ad-ranking and bigger-model work, and drops it within milliseconds when normal serving needs the capacity back. story 3 ↓
Anthropic's "flows" let Claude write its own multi-agent orchestration. In a demo they caught more but cost "much more" than one agent. story 4 ↓
Cloudflare embeds DataFusion in its Go log pipeline to run customer SQL filters. If a transform fails, delivery stalls instead of sending raw logs. story 5 ↓
1/5PlanetScale· Ahmed Darwich· Oct 12
How Postgres compresses large values, and why version 19 switches to LZ4
PlanetScale engineer Ahmed Darwich explains when Postgres compresses large values or moves them out of the row, and tests its two compressors. LZ4 is faster; the older pglz sometimes compresses smaller or reads previews faster.
Why it matters · Postgres 19 changes a storage default that affects disk use, memory hit rates and read latency, and the trade-offs explain when to override it by hand.
Know this first
Context from the editor, not the article · 35 sec
1 · Postgres pages and rows
Postgres keeps table data in fixed 8 KB blocks called pages. A row must fit inside one page. Small values sit directly in the row. Very large text, JSON or binary values need special handling to fit, and that handling is what this story is about.
2 · Back-reference compression
Most fast compressors scan the data and replace a repeated run of bytes with a short note: "copy N bytes from X bytes back". How far back the compressor can look is its window. A repeat that sits farther back than the window can't be found, so it gets stored in full.
3 · Defaults versus per-column settings
2/5Engineering at Meta· Oct 12
Instagram Live switches its video upload path to Media over QUIC, an open standard
Meta now carries Instagram Live uploads from phones over Media over QUIC, an open streaming protocol still in draft. Its goal was to match the old in-house protocol without regressions, and it did.
Why it matters · It shows how to replace a transport protocol under a large live service without users noticing: translate at the edge, route on the server side, and make every step reversible.
Know this first
Context from the editor, not the article · 39 sec
1 · Ingest versus playback
A live stream has two halves. Ingest is the upload from the creator's phone to the company's servers. Playback is the delivery from those servers to viewers. The two halves have usually used different protocols, and each protocol comes with its own servers, monitoring and specialist engineers.
2 · TCP, UDP and QUIC
TCP delivers bytes strictly in order, so one lost packet holds up everything behind it. This is called head-of-line blocking. QUIC runs on UDP, a simpler 'send and forget' protocol. QUIC adds encryption and many independent streams, so a loss on one stream doesn't stall the others. QUIC is also the transport under HTTP/3.
3 · How internet standards evolve
3/5Engineering at Meta· Oct 12
Meta uses idle inference hardware to rank more ads and run bigger models
Meta pools idle machine-learning serving hardware from across its fleet and uses it for optional ads work: scoring more candidate ads, or sending more requests to a stronger model. When the hardware is reclaimed, that work shrinks automatically.
Why it matters · It's a reusable pattern for running optional work on hardware that can be taken back at any moment: split work into required and optional, isolate them, steer with feedback, and shed load fast.
Know this first
Context from the editor, not the article · 39 sec
1 · What inference capacity is
Training builds a machine-learning model. Inference runs the finished model on live requests. Large companies keep fleets of servers with accelerators (GPUs or custom chips that speed up model math) for inference. They size these fleets for peak traffic and strict response-time targets, so some hardware sits unused at quieter times.
2 · How ad ranking works
To show a user a few ads, the system starts from millions of candidates and narrows them in stages. Cheap stages discard most candidates, and costlier, more accurate models score only the survivors. Before that, CPU-heavy steps fetch candidates and compute their input features. The whole pipeline must finish in under a second.
3 · Preemptible capacity
4/5Latent Space· Richard MacManus· Oct 12
Anthropic's new "flows" let Claude design and run its own agent teams
Anthropic added "dynamic workflows" to its hosted agent service. Developers no longer have to design a team of AI agents by hand: Claude plans the work, writes a program that launches helper agents, and gathers their results.
Why it matters · The work of designing multi-agent systems is moving from engineers to the model itself, which changes where engineers keep control: in budgets, evals and inspecting what the agents did.
Know this first
Context from the editor, not the article · 37 sec
1 · What an AI agent is
An agent is a language model running in a loop: it reads a task, calls tools (run code, search, edit files), looks at the results and decides what to do next. Everything it can see at once sits in its context window, which has a size limit, and every word it reads or writes costs tokens, the unit labs bill by.
2 · Why use many agents
One agent working on a big job fills its context and slows down. Multi-agent orchestration splits the job: a lead agent hands pieces to subagents, each with a fresh, isolated context, and then combines what they find. Until now, developers usually wired up that structure themselves.
3 · Claude Managed Agents
5/5Cloudflare· Rob Taylor· Oct 12
Cloudflare runs customer SQL inside its log-export pipeline to filter logs
Cloudflare now lets customers write a SQL query that filters, redacts and reshapes logs before Logpush exports them. The query runs inside the Go delivery service on an embedded Rust engine, with no extra network hop.
Why it matters · It shows how to put user code on a very high-throughput hot path: run it in-process, pass columnar batches between languages, and keep memory ownership explicit.
Know this first
Context from the editor, not the article · 40 sec
1 · What log export is for
Services like Cloudflare produce a log record for every web request. Customers ship these to a SIEM (security information and event management system), an analytics tool or cloud storage. Destinations often charge per byte ingested and stored, so useless records such as health checks cost money, and sensitive fields like IP addresses can create compliance problems.
2 · Row versus columnar data
A row format like JSON stores each record whole. A columnar format stores each field as its own contiguous array: all timestamps together, all status codes together. Query engines prefer columns because a CPU can scan one tight array very fast. This is called vectorized execution.
Postgres has a server-wide default compressor (default_toast_compression), and a single column can override it with a COMPRESSION clause. Changing either setting affects only values written afterwards. Values already stored keep the compressor they were written with.
Image: PlanetScale
Postgres 19, now in beta, switches its default compressor for large column values from 1, Postgres's home-grown algorithm, to 2, a faster general-purpose one (in builds that include LZ4). PlanetScale, a hosted database company, already made the switch on Postgres 17 and 18. Ahmed Darwich's post explains the mechanism behind that default and measures where each choice wins.
Compression starts at a threshold
Postgres stores rows in 8 KB pages, and a row can't span pages. 3 (The Oversized-Attribute Storage Technique) steps in only when a row passes 2,032 bytes, about a quarter of a page, which keeps at least four rows per page. A row of an int plus 2,000 x characters is exactly 2,032 bytes (24-byte header + 4 + 2,004), so it's stored uncompressed. Add one character and LZ4 shrinks the text to 26 bytes.
Once a row is over the threshold, TOAST works on its variable-length values (text, jsonb, bytea), largest first. It can do two things:
Compress a value and keep it in the row.
Move it out to a hidden 4, cut into chunks of up to 1,996 bytes, leaving an 18-byte pointer behind.
A column's storage strategy decides which of these TOAST may do. The default for text-like types, EXTENDED, allows both. EXTERNAL allows moving out but never compressing.
pglz versus LZ4
Both compressors replace repeated bytes with back-references. pglz looks back only 4 KB but searches several candidates for the longest match, and Postgres discards its result unless it saves more than 25%. LZ4 looks back 64 KB and checks a single candidate, which makes it much faster.
On 210 KB of random words that repeat about every 35 KB, pglz gave up and stored the data raw, while LZ4 saved about 15%. On 10,000 JSON order documents, pglz compressed better (19 MB versus LZ4's 24 MB, from 103 MB raw). LZ4 loaded them faster: 0.48–1.09 s against 1.59–1.73 s.
pglz also wins on previews such as left(body, 200). Its output size has a known upper limit, so Postgres can fetch just the first chunk. On 100 values of about 1 MB, that took about 1 ms, against about 16 ms for LZ4, which has to fetch the whole value.
When not to compress
Darwich recommends EXTERNAL for data that's already compressed, such as JPEGs, and for reading slices from the middle of large bytea (raw binary) values. Uncompressed chunks map to fixed byte offsets, so Postgres fetches only the chunk it needs. On the same 100 values, a 200-byte read at offset 500,000 took 0.6 ms with EXTERNAL, 43 ms with LZ4 and 116 ms with pglz.
His verdict: LZ4 is the right default for most tables, with pglz or EXTERNAL chosen by hand for these cases. The benchmarks ran on PlanetScale's Postgres 18, and the load timings come from only two runs each.
The news, and a one-byte experiment
Postgres 19, now in beta, changes how the database compresses large column values. In builds compiled with 1 support, the default switches from 2, Postgres's own compressor, to LZ4, a fast general-purpose compressor used well beyond Postgres. PlanetScale, a hosted database company, had already made that switch on Postgres 17 and 18. In a post on its blog, Ahmed Darwich explains the machinery behind the default and measures where each choice comes out ahead.
His opening experiment: take a table with an integer ID and a text column. Insert 2,000 copies of the letter x and Postgres stores 2,004 bytes, uncompressed. Insert 2,001 and Postgres compresses the value with LZ4 down to 26 bytes. To see why one byte makes that difference, you need to know how rows are laid out.
Where a row's bytes go
Postgres stores tables in 8 KB pages, and a row can't be split across pages. Every row begins with a 24-byte header that tracks which transactions are allowed to see it. The column values come after the header.
Fixed-width types always take the same space: bigint takes 8 bytes, timestamptz 8 and boolean 1.
Variable-length types such as text, jsonb (binary JSON) and bytea (raw bytes) take as much space as the value needs, plus a length header. That header is 1 byte for values up to 126 bytes and 4 bytes for anything longer.
So the row (1::bigint, now(), true, 'Ada') is 45 bytes: 24 for the header, 17 for the three fixed-width values, and 4 for "Ada" (three characters plus a 1-byte header). Only variable-length values can grow without limit, and only they can be compressed.
The threshold
3 (The Oversized-Attribute Storage Technique) is the part of Postgres that handles oversized rows. A row counts as oversized once it passes 2,032 bytes, about a quarter of a page. That figure guarantees at least four rows fit on a page once the page header and the per-row pointers are counted. The 2,000-character row comes to exactly 2,032 bytes (24 + 4 + 2,004), so TOAST leaves it alone. The 2,001-character row is one byte over.
TOAST's two jobs
The threshold applies to the whole row, but TOAST acts on individual values. It has two tools:
Compress a value and keep the smaller version in the row.
Move it out to a separate 4. Postgres creates one automatically in the pg_toast schema for any table whose columns can hold large values.
A moved-out value is cut into chunks of up to 1,996 bytes. Each chunk is stored as its own row, tagged with the value's ID (chunk_id) and its position (chunk_seq), and four chunk rows fit on a page. The original row keeps an 18-byte pointer to the chunks. TOAST works through a row's values from largest to smallest and stops as soon as the row is back under the target (toast_tuple_target, 2,032 bytes by default).
Darwich stored four values in one LZ4 table, and each ended up in a different place:
Value
Raw bytes
Stored bytes
Outcome
200 × x
200
204
untouched
abc repeated
10,200
60
compressed, stays in row
random text
200,000
200,000
moved out, uncompressed
repeated sentence
1,125,000
4,471
compressed and moved out
Each column has a storage strategy that decides which jobs TOAST may do:
PLAIN: neither job. Used by fixed-width types.
EXTENDED: compress, then move out if needed. The default for text, jsonb and bytea.
EXTERNAL: move out, never compress.
MAIN: compress, and keep the value in the row as long as the row still fits on a page, moving it out only as a last resort. The default for numeric.
ALTER TABLE ... SET STORAGE changes the strategy for future writes only. TOAST is also different from Postgres's large objects, which you create explicitly to store whole files. TOAST values top out at 1 GB; large objects can hold up to 4 TB. TOAST values are deleted along with their row, while a large object stays until you remove it with lo_unlink.
How the two compressors differ
Both compressors scan a value and replace repeated bytes with a short back-reference. In {"status":"active"},{"status":"active"}, the second object becomes a few bytes that point back at the first. They differ in three ways: how far back they can look, how hard they search, and when Postgres accepts the result.
pglz looks back up to 4 KB. At each position it checks several earlier matches and keeps the longest. Postgres throws the result away unless it saves more than 25%, and pglz gives up early on data with no repeats.
LZ4 (supported since Postgres 14) looks back up to 64 KB, but it checks only one candidate, taken from a small lookup table that keeps overwriting older entries. That makes it much faster, but the farther back a repeat sits, the less likely LZ4 is to find it. Postgres keeps an LZ4 result if it saves more than 2 bytes.
zstd, another popular compressor, isn't available for columns. Postgres uses it only elsewhere, for example to compress the write-ahead log (5), the record of changes that gets replayed on replicas. Existing values keep their compressor even through INSERT ... SELECT or VACUUM FULL.
To test the window, Darwich built a 210 KB string from 30,000 random six-letter words:
500-word vocabulary (a repeat every 3.5 KB on average): pglz stored 105,457 bytes and LZ4 96,022, about 9% smaller. Roughly 30% of the repeats fell beyond pglz's 4 KB reach.
5,000-word vocabulary (a repeat every 35 KB): pglz couldn't save 25%, so the value was stored raw at 209,999 bytes. LZ4 got it down to 177,590, about 15% smaller.
Where pglz wins
Repetitive JSON. Darwich loaded 10,000 order documents with 25 to 100 line items each, all sharing the same five keys. Uncompressed they took 103 MB; pglz brought that to 19 MB and LZ4 to 24 MB. Both compressors find the nearby repeated keys, so pglz's longer matches win on size. LZ4 loaded faster: 0.48 s and 1.09 s over two runs, against 1.59 s and 1.73 s for pglz.
Previews. pglz never needs more than 9 bits per original byte plus 2 bytes overall, so Postgres can bound how many compressed bytes a prefix needs. Take left(body, 200): 200 characters are at most 800 bytes, which need at most 902 compressed bytes. That fits inside one 1,996-byte chunk, so Postgres fetches only the first chunk. LZ4 offers no such bound, so Postgres fetches the whole value. On 100 values of about 1 MB each, pglz took 301 page lookups (about 1 ms) and LZ4 took 6,234 (about 16 ms). Uncompressed EXTERNAL storage also took 301, but used about twice the disk.
When to skip compression
Compression means fewer pages, less disk, more of the table cached in memory, and less WAL to ship to replicas and backups. The cost is CPU on every write past the threshold and every read of a compressed value. It doesn't reduce network egress, because Postgres decompresses values before sending them to the client. Darwich recommends EXTERNAL in two cases:
Data that's already compressed, such as JPEG frames. LZ4 spends CPU and saves little or nothing.
Reads from the middle of large bytea values. Uncompressed, every chunk but the last holds exactly 1,996 bytes, so Postgres can calculate where any byte lives. The first 250 chunks cover 499,000 bytes, so a 200-byte read at offset 500,000 needs only chunk 250. Compressed, Postgres must decompress from the beginning. Measured on the same 100 values: 0.6 ms with EXTERNAL, 43 ms with LZ4, 116 ms with pglz. This shortcut works only for bytea. For text in a UTF-8 database, characters vary from 1 to 4 bytes, so Postgres has to read from the start.
What stays open
The article's closing list of recommendations is cut off in the text available here. Its overall verdict is that LZ4 is the right default for most tables, because its speed matters more than a size gap that can go either way. The load timings come from only two runs each, and the LZ4 runs differ by more than 2×. The benchmarks ran on PlanetScale Postgres 18 with unstated hardware, and full-value decompression speed wasn't measured separately.
Remember
Values in rows under 2,032 bytes are never compressed, however repetitive they are.
Changing the compressor or storage strategy affects only new writes. Existing values keep theirs, even through VACUUM FULL.
Compression saves disk, memory and replication traffic but not network egress, because Postgres decompresses values before sending them.
The IETF (Internet Engineering Task Force) writes the core internet protocols. A protocol goes through numbered drafts that can change in incompatible ways before it becomes a final RFC. Anyone who ships against a draft has to pin a specific version and handle clients that speak older ones.
Meta's chart showing MoQ covering the whole latency range, from real-time calls to buffered streaming.
The news
Instagram Live on iOS and Android now uses 1 for ingest, the upload from a creator's phone to Meta's servers. MoQ is an open streaming protocol being drafted at the 2 (Internet Engineering Task Force). It replaces 3, a protocol Meta built in-house and ran for nearly ten years. The goal was parity, not improvement. After months of experiments, Meta says MoQ matched RUSH on every quality and reliability metric it tracked. The post doesn't give the numbers.
Why switch
Live video today runs on a patchwork of protocols:
4 for uploading
5 for playback (downloading short video segments over HTTP)
6 for calls
RUSH for whatever didn't fit the others
Meta says one user going live and then making a voice call passes through three separate stacks, each with its own servers and monitoring. Meta wants one protocol for upload, delivery and playback. It also wants a shared standard that other companies help improve, instead of a protocol only Meta maintains.
How MoQ works
MoQ runs on 7, a transport whose independent streams don't stall each other when a packet is lost. It also survives a phone switching networks. Media travels as tracks: named streams of objects that publishers push and subscribers receive through 8. Relays cache and forward objects without decoding the video. A relay can fetch an object once and send it to many subscribers. It can also drop or prioritize objects using their metadata. Each subscriber can choose its own delay, from about 100 ms to several seconds.
How they swapped it safely
Client: the apps use 9, Meta's open-source MoQ library. A single config flag chooses MoQ or RUSH for each broadcast.
Edge translation: edge servers convert MoQ into RUSH format, so the backend didn't change. This shim is temporary.
Server-side routing: the system that hands out stream URLs decides which broadcasts use MoQ. The rollout can grow or shrink in minutes without waiting for app updates.
There were snags. The draft spec kept changing, so Meta pinned specific versions. Some corporate and carrier networks block QUIC's traffic, so each stream races a QUIC connection against a TCP one and keeps the winner. Encryption also hid the protocol's framing from Meta's own debugging tools, so the team added MoQ-aware tracing.
What's still open
The gains so far are strategic, not visible to users. Lower latency would come from using MoQ for playback and from letting players skip stale video after a loss, and neither has shipped yet. The backend and the TCP fallback still depend on RUSH.
What shipped
Instagram Live on iOS and Android now uses 1 for ingest, the upload from a broadcaster's phone to Meta's servers. MoQ is a streaming protocol being standardized at the 2 (Internet Engineering Task Force), with Meta, Google, Akamai, Cisco, Cloudflare and academic groups taking part. Meta says this is one of the largest uses of MoQ it knows of, measured by number of ingest streams and total volume.
The result was meant to be modest. The goal was parity: the new path had to match the old one on every quality and reliability metric used in the rollout, and Meta says it did. The post gives no numbers.
Why replace something that works
Today's live-video stack is a patchwork of protocols, each built for one job:
3 (Real-Time Messaging Protocol), from the Flash era, is still common for uploading streams.
4 handle playback by having viewers download short video segments over ordinary HTTP.
5 handles real-time calls.
6 (Reliable Ultra-low-latency Streaming over HTTP) is Meta's own QUIC-based protocol. It replaced RTMP for Meta's live ingest nearly ten years ago.
According to Meta, a user who starts a live stream and then makes a voice call in the same app passes through three separate protocol stacks. Each has its own servers, monitoring and operating expertise. Meta argues that live video, real-time AI and generative creation tools are converging on the same product surfaces, so one protocol for ingest, delivery and playback would cut the number of systems to maintain.
There is a second reason. RUSH worked well, but only Meta evolved it, and it was "falling behind the open ecosystem." With a standard, several implementers can test against each other and improve one shared specification.
How MoQ works
MoQ is built around the track: a named, addressable stream of media objects, such as the video of one broadcast at one quality. Publishers push objects into a track. Subscribers receive them through a network of 7. Relays cache and forward objects without decoding the video. They read only MoQ's framing and delivery metadata. A rough analogy: a postal sorting centre that reads the envelope and never opens the letter.
Meta highlights these properties:
How the rollout worked
Meta says swapping a transport is hard for reasons that have little to do with the protocol itself. It split the work into three constrained problems:
Clients. Meta built a MoQ publisher on 13, its open-source MoQ library, and added it to four apps (the post doesn't name them). Moxygen can act as publisher, subscriber or relay. It sits on , Meta's QUIC implementation. A single config parameter chooses MoQ or RUSH for each broadcast.
Operational hurdles
A moving spec. The MoQ draft kept changing during the build. Meta pinned specific draft versions, kept compatibility shims for older clients, and contributed fixes back to the working group.
Firewalls. UDP on port 443 is still blocked or rate-limited on what Meta calls a "non-trivial slice" of corporate and mobile-carrier networks. At the start of each stream, the client tries QUIC and TCP handshakes in parallel and uses whichever completes first for the whole session. For now the TCP path uses RUSH's existing fallback.
Debugging. QUIC encrypts its framing, so neither network middleboxes nor Meta's own packet captures can read it. Meta added MoQ-aware tracing at the edge to log object-level events without decrypting traffic.
Results
After months of experiments on Instagram for iOS and Android, MoQ matched RUSH on the streaming-quality and reliability measures used for the rollout, and Meta launched it. The post doesn't report latency figures, stall rates, error rates or the share of traffic involved.
What's next, and what's still open
Meta plans to extend MoQ to the rest of its apps. It wants to use subgroups, a MoQ construct that carries audio, video and metadata on independent streams, so a loss in one doesn't stall the others. It also plans to adopt QMUX for TCP fallback, which would remove the last dependency on RUSH and let Meta retire it. Further out:
End-to-end MoQ for playback, where media is pushed as soon as it's produced.
Partial reliability, so a player can skip stale video after a loss instead of falling further behind.
Real-time conversational AI, which needs a single low-latency, two-way channel for both control messages and media.
Today's benefit is mostly strategic: a production-tested transport, rollout controls and observability. Users don't see it yet. The latency gains Meta describes depend on playback and partial reliability, and neither has shipped. The backend still runs on RUSH, the TCP fallback still runs on RUSH, and the spec is still a draft. Without published numbers, readers have only Meta's word that the two paths performed the same.
Remember
Meta made the new protocol a drop-in by translating MoQ into RUSH at the edge, leaving the backend untouched.
Choosing the protocol on the server, through the stream URL, let Meta ramp up or roll back in minutes without app updates.
Meta's latency claims depend on moving playback to MoQ and on partial reliability. Neither has shipped yet.
Preemptible capacity is compute you may use only until its owner needs it back, sometimes with almost no warning. Cloud spot instances are the familiar example. Work placed there must tolerate losing its machines abruptly without harming anything that depends on reserved, guaranteed capacity.
Preemptible
Usable only until the owner reclaims it, possibly with no warning.
Tail latency
Response time of the slowest requests, for example the 99th percentile. It is usually the first measure to worsen under load.
Explore-and-exploit
A strategy that mostly uses the best-known option while still testing alternatives, so its estimates stay current.
Load shedding
Deliberately dropping some work when a system is overloaded, to protect the work that matters most.
Overview of the system: isolated routing, the two feedback loops and the layered load shedding that protect baseline serving.1 / 2
Meta's engineering team has described a system that collects idle inference hardware from across its fleet and uses it to improve ad recommendations. It does this without ordering new machines and without slowing normal serving. 1 means running trained models on live requests.
The paradox
Meta calls the starting point a capacity paradox: the fleet as a whole is short of capacity, yet pockets of it sit idle. The post gives four causes:
The fleet is sized for peak traffic, so hardware sits unused at quiet hours.
New hardware arrives before the models that would use it are ready.
2 (chips specialised for model math) and CPUs aren't available in the right ratio in a given region.
Some models can't run on some chip types.
Meta gathers these pockets into a shared pool. The pool is preemptible, meaning the owning teams can reclaim it at any time, and it comes with no guarantees.
Two levers
Work that runs on this pool has to be able to grow or shrink a lot, and quickly. Meta found two ad workloads that fit.
3 sends more candidate ads through the ranking stages, which narrow millions of candidates down to a few. More candidates means a better chance of finding a relevant ad. The extra candidates also pass through CPU-heavy retrieval and feature processing, so RMA needs spare CPUs as well as spare accelerators.
4 sends a larger share of requests to a more powerful but costlier model. The number of candidates stays the same, so SBM can use spare accelerators even when no CPUs are free.
Because the two levers need different mixes of hardware, together they can use more of the pool than either could alone.
Keeping baseline traffic safe
The rule is that losing spare capacity may remove optional work, but it must never overload the reserved hardware.
SBM isolates physically. The powerful model runs only on spare machines. When those shrink, requests fall back to the baseline model, whose capacity is already reserved for them.
RMA isolates logically. A global budget caps the extra ranking work at what the spare replicas can absorb.
Two control loops steer the levers. For RMA, a 5 (a standard feedback algorithm) watches tail latency, the response time of the slowest requests. It raises candidate counts while latency stays under target and lowers them when it rises. For SBM, a central controller sets the powerful model's coverage, and a router picks the requests likely to benefit most.
These controllers react in minutes, but capacity can disappear in seconds. To bridge that gap, both servers and clients drop the lowest-value optional work within a few milliseconds of seeing overload.
What's missing
The post reports no numbers: no capacity reclaimed, no gains in ad relevance or revenue. Meta is now building an optimizer that would decide automatically how to split capacity between the levers.
Meta's engineering team has described a production system that collects idle inference hardware from across its fleet into a shared pool and uses it to improve ad recommendations. 1 is running trained models on live requests. The system doesn't require new hardware orders. Its main design rule is that optional work on borrowed hardware must never slow the "baseline" traffic that runs on reserved hardware.
Why a constrained fleet has idle hardware
The post opens with what it calls the inference capacity paradox: a fleet can be short of capacity overall while meaningful pockets of it sit idle. Meta lists four causes:
Peak provisioning. The fleet must be sized for the busiest hour and season, so much of it idles off-peak.
Timing. Hardware arrives on supply-chain schedules, while models launch on their own. Capacity often lands before the models that would use it are ready.
Resource ratios. Serving needs accelerators (GPUs or custom chips specialised for model math) plus supporting CPUs. If a region has plenty of one and too little of the other, the surplus resource sits idle.
Accelerator types. The fleet mixes vendors, chip generations and architectures. A model built for one chip may not run on another, which strands that hardware.
Meta says better planning can't remove this, because in a fleet this large and varied some momentary idleness is inevitable. The leftover capacity is pooled across teams' allocations, regions and chip types. It has awkward properties: it is preemptible (the owning team can reclaim it at any time), its availability swings widely, and it carries no long-term guarantee.
What kind of work fits
Meta names three properties a workload needs to run on this pool:
Elasticity. It can add or drop large amounts of optional work quickly.
Quality that keeps improving. More compute should keep buying better results across the whole range the pool can supply.
Few dependencies. It should be self-contained, so that few upstream or downstream systems have to react when capacity swings.
Ad ranking fits. To pick a few ads from millions in under a second, the system runs a sequence of stages. Each stage uses a more accurate and more expensive model on a smaller set of survivors from the stage before.
Lever one: Rank More Ads
2 widens that funnel when spare capacity exists, sending more candidates into the ranking stages. More candidates means a better chance of surfacing a relevant ad. The extra candidates also pass through CPU-heavy steps before any model runs, such as retrieval (fetching candidates) and feature processing (computing model inputs). So RMA needs spare CPUs and spare accelerators at the same time.
Lever two: Serve Better Models
3 leaves the candidate count alone and changes which model scores them. Meta has more powerful models that predict better but cost too much to run on every request using reserved hardware. SBM sends a variable share of requests to the stronger model as accelerator supply changes. The number of candidates doesn't change, so upstream CPU load stays roughly flat. That means SBM can use accelerator-only surplus that RMA cannot.
Each lever covers gaps the other leaves. An illustrative example (not from the post): a region with idle GPUs but busy CPUs can only feed SBM, while a region with both spare can feed either. Meta periodically rebalances the two based on measured value and resource needs.
Isolation: physical and logical
The governing rule is that losing borrowed capacity may cut optional work but must never spill load onto reserved hardware. The two levers enforce this differently.
SBM is physically isolated. The baseline model and the powerful model have separate routing targets. Reserved replicas are sized to serve every request with the baseline model, and only replicas on borrowed hardware run the powerful model. When the pool shrinks, fewer requests are routed to the powerful model, and the rest fall back to the baseline model, whose capacity was reserved for them all along.
RMA is logically isolated. Individual replicas handle a mix of baseline and extra work. The system calculates how much extra candidate-ranking work the replicas on borrowed hardware can support and sets a global budget no larger than that. Baseline traffic stays safe because the total extra work is capped.
Feedback loops
RMA uses a 4. PID stands for proportional-integral-derivative, a classic control algorithm that adjusts an output based on the current error, its accumulated history and its rate of change. The controller watches tail latency, the response time of the slowest requests. It computes a target for how many ads to send to ranking, and clients apply that target to new requests. When latency is under target, the count rises; when latency climbs, it falls. Latency acts as an indirect measure of how much capacity is available.
SBM has a central controller that sets target coverage for the powerful model, and a client-side router that chooses which requests get it. The router groups requests into segments by predicted likelihood that the user engages with the ad. An explore-and-exploit process, the same idea as a multi-armed bandit (an algorithm that balances trying options with using the best one known), measures how much the powerful model actually helps in each segment and shifts allocation to match. Meta notes that the most valuable requests aren't always the ones that gain most from a better model, which is why the gain has to be measured rather than assumed. The process always keeps some exploratory traffic in every segment so the router doesn't lock into a stale decision. A separate calibration loop keeps the actual traffic share close to the target as the mix shifts across regions and times of day.
Fast shedding for sudden loss
Both central controllers work on a scale of minutes, and capacity can vanish faster than that. Two layers of load shedding (deliberately dropping work under overload) fill the gap:
Server-side. A server that loses capacity before clients have caught up drops lower-value optional work immediately. For RMA, it scores fewer candidates within a request. For SBM, it applies the same kind of value-based policy to requests routed to the powerful model.
Client-side. Clients follow controller targets but also watch server status directly. On seeing overload, RMA clients request fewer extra ads and SBM clients route fewer requests to the powerful model.
Both layers react within a few milliseconds, and both use value signals to choose what to drop, which keeps the quality cost of the transition low.
What's open
The post contains no quantitative results. It says the system drives "significant advertiser value and capacity efficiency" but gives no figures for how much capacity is reclaimed, no relevance or revenue gains, no preemption rates and no incident history. Readers can't judge the scale of the benefit. Meta lists three next steps: widening each lever's range, including through precomputation (doing work ahead of requests, so it's less tied to where and when live traffic arrives); improving model portability and cross-region inference so more of the stranded hardware becomes usable; and an optimizer that automatically splits capacity among levers, subject to region, hardware-compatibility and durability constraints.
Remember
Treat spare capacity as a place for optional work only, so losing it can never spill load onto reserved serving.
Pairing levers with different CPU and accelerator needs lets the system use more kinds of stranded hardware.
Controllers that react in minutes need millisecond load shedding behind them, because borrowed capacity can vanish faster than the controllers adjust.
Anthropic's hosted service for running Claude agents through its API (the programmatic interface developers call). Developers configure agents in Claude Console, Anthropic's web dashboard, and Anthropic runs the loop, the tools and the memory for them.
Anthropic's diagram: a lead agent plans, then Claude writes an orchestration program that launches subagents.1 / 4
Anthropic has added dynamic workflows, or "flows," to 1, its hosted service for running AI agents. On Monday, two Anthropic engineers, CJ Avilla and Harrison Stall, demonstrated the feature at the AI Engineer New York conference.
The problem
An agent is a language model running in a loop: it takes a task, calls tools such as code execution or search, and decides what to do next. Big jobs often get split across many agents. Until now, the developer had to design that split by hand: a lead "coordinator" agent plus a fixed roster of helper 2. Avilla described the result: engineers become "the bottleneck," stuck reviewing endless pull requests (proposed code changes) from their agents.
How flows work
With flows, Claude designs the team. The lead agent writes a plan. Claude then writes an 3, real code that launches subagents, collects their results and routes information between them across several phases. Stall called this letting Claude "fully orchestrate the entirety of its own multi-agent 4." Developers can still supply pre-built specialist subagents if they want.
The demo was a stock watchlist scanner. In a scan phase, each ticker went to its own agent with an isolated context (a separate working memory), so one company's research didn't clutter another's. A review phase then checked the findings and wrote a report. According to Avilla, this cost "much more" than a single agent but was "much more effective at finding all of the details." The talk gave no exact figures.
Keeping control
The audience questions brought out the practical limits:
Agent count: Claude is tuned toward low defaults through 5 (behavior tests), so big parallel runs usually need explicit prompting.
Budget: developers can set a session spending cap, but the model isn't told about it. Stall joked that telling it would make agents "a little bit anxious."
Communication: subagents share a filesystem but can't message each other directly.
Visibility: Claude Console, Anthropic's developer dashboard, shows each run as a graph, including the exact data passed to each subagent.
To make repeated runs cheaper, Avilla pointed to "6," a preview feature that reads past session transcripts and rewrites the agent's stored memory, merging duplicates and dropping stale entries. Flows can't yet rewrite their own approach between runs.
The pitch, and the gaps
Stall described flows as "a way to throw 7 at hard problems," meaning more computation spent while answering, and said the aim is to put large-scale problem-solving within reach of ordinary API users. The talk left a lot unproven: there were no benchmarks against hand-built agent teams and no cost multipliers, and the demo was far smaller than the "ambitious workloads" Anthropic is pitching flows for.
On Friday Anthropic announced a feature for 1, its hosted service for running AI agents, called dynamic workflows, or "flows" for short. Anthropic describes it as "a new type of multiagent orchestration, built for your most ambitious workloads." On Monday, two Members of Technical Staff, CJ Avilla and Harrison Stall, walked through it at a workshop at the AI Engineer New York 2026 conference. Most of what follows comes from that session and its Q&A, as reported by Richard MacManus for Latent Space. It is a conference write-up, not a technical paper, so it has few numbers.
The problem: the human becomes the bottleneck
The presenters framed it with the history of operating systems. Ken Thompson wrote the original Unix largely on his own. Linux started with one person, Linus Torvalds, and grew into layers of trusted maintainers who each review part of the code. Their point was that large projects need organization, and that groups of AI agents need it too.
Avilla described a feeling that engineers who run many coding agents will recognize: "being the bottleneck and being the breaking point as we have lots of agents contributing tons and tons of PRs." A PR, or pull request, is a proposed code change that someone has to review and approve. When the agents produce work faster than a person can check it, adding more agents doesn't help unless something else takes over the coordination.
What changed: Claude picks the team structure
Before flows, Managed Agents asked the developer to set up the structure. You configured a coordinator (a lead agent) and a roster of 2, helper agents with their own instructions, and decided how they connected. The idea grew out of 3, Anthropic's command-line coding agent, where it went by the slogan "a 4 for every task." A harness is the scaffolding of prompts, tools and control logic wrapped around a model.
With flows, Claude designs that scaffolding. In Stall's words, flows let "Claude fully orchestrate the entirety of its own multi-agent 5". Topology here means which agents exist and how information moves between them. The mechanism works in three steps:
The lead agent writes a plan for the task.
Claude then writes an 6: actual code that launches agents, collects their results and controls what information goes where.
That program runs across several phases, each producing what Stall called "discrete, verifiable units of work," and the developer doesn't have to manage how the pieces connect.
Developers can still pass pre-configured specialist subagents in the agent config if they want particular roles. Flows can also run longer than a single agent or a fixed roster.
A worked example: a stock watchlist scanner
The demo was deliberately small. Given a watchlist of stock tickers, the flow had two phases:
Scan phase. Each ticker went to its own agent with an isolated context, so research on one company couldn't clutter or bias the work on another. As an illustration, a ten-ticker list could mean ten parallel scanners.
Review phase. A later step went over all the findings and produced a single report.
This fan-out-then-review pattern is the basic shape. Avilla said the flow cost "much more" than a single agent doing the same job, but was "much more effective at finding all of the details." The article gives no actual cost figures or accuracy numbers.
Control: how many agents, and at what cost
Most of the useful material came from audience questions.
Could a simple scan spawn hundreds of agents? Stall's answer was "7 and lower defaults." Evals are test suites that measure model behavior. Anthropic tunes Claude to be conservative by default, so ambitious tasks generally need explicit prompting to parallelize more.
Does the lead agent know its budget? No. Developers can set a session budget, a hard spending cap, but the model isn't told about it while it runs. Stall said telling models their budget could make them "a little bit anxious." The likely implication is that the model might cut corners, though the article doesn't spell this out. In practice, the cap is your safety net, and Anthropic's own slide says setting one "is key."
Can agents talk to each other? Only indirectly. Subagents share a filesystem and can leave information there for each other, but the API has no way to tell one subagent to message another.
Can you see what happened? Yes. Claude Console shows the run as a graph, and you can inspect the payload, meaning the exact data, the lead agent passed to each subagent at every step.
Getting cheaper over time
Asked how repeated flows could become more efficient, Avilla suggested asking Claude to review its own costs, and also pointed to 8, a research-preview feature. According to the documentation, a dream "reads an existing memory store alongside past session transcripts, then produces a new, reorganized memory store." It merges duplicates, replaces "stale or contradicted entries," and surfaces "new insights." Avilla described this as a way to "crystallize" what past runs learned.
Could the watchlist flow notice that its scanning approach is weak and change its next scheduled run? Stall said that isn't built in. The presenters suggested combining dreaming with 9, which means making small changes, keeping the ones that score better, and repeating.
The bigger claim
An audience member asked whether flows could take on hard science, citing OpenAI's recently announced proof concerning the Navier–Stokes Millennium Prize problem. Stall wouldn't discuss Anthropic's internal methods. He said "flows are basically a way to throw 10 at hard problems". Test-time compute means spending more computation while answering, rather than during training. He added that offering this through the API is meant to make that kind of work "broadly doable by the general public."
What's left open
The talk did not show how good Claude's self-designed topologies are compared with hand-built ones. It gave no cost multipliers, no benchmarks, and no details on how failures in one subagent are handled. Communication only through a shared filesystem leaves open how agents avoid overwriting each other's files. The demo was much smaller than the "ambitious" work flows are pitched for. For now, the sensible approach is to set a budget, use the graph view, and measure flows against a single agent on your own tasks.
Remember
Developers no longer hand-wire agent rosters; Claude writes code to orchestrate its own subagents.
Session budgets are hard caps the model never sees, so setting one is your main cost control.
Subagents coordinate only through a shared filesystem, not direct messages.
Go and Rust each manage memory their own way. They can call each other only through a C-style interface, which Go reaches with a tool called cgo. Each cgo call has a cost, and if both sides think they own the same memory, it can be freed twice or never freed at all.
Data flow: Buftee feeds Go-based Bufstream, which sends Arrow batches to Rust DataFusion and exports NDJSON.
Cloudflare has made Transformers generally available for 1, its service that exports customer logs to outside tools in near real time. A Transformer is a SQL query that runs on each batch of logs before export. WHERE decides which records are kept, and SELECT decides what each exported record looks like.
Why it exists
Teams send Cloudflare logs to 2 (security information and event management systems), analytics platforms and storage. Along the way they often want to drop health-check noise, redact sensitive fields such as part of a client IP, or rename and nest fields to match a shared schema. Doing that usually means running a separate processing service. Teams also pay to ship and store records they immediately throw away. Transformers move that work into Logpush. Output is 3, one JSON object per line.
Why it's hard
4, Cloudflare's Logpush delivery service, moves more than 110 million records per second across hundreds of Kubernetes pods. A few jobs alone push millions of records per second. The first prototype ran transforms in Cloudflare Workers, its serverless platform. That added a network call to every export, along with the timeouts and retries that come with one. Cloudflare dropped it and ran the SQL in-process instead.
How it works
Bufstream is written in Go. The SQL engine is 5, written in Rust. DataFusion is built on 6, a format that stores data column by column so CPUs can scan it quickly.
For each batch, Bufstream:
decodes the logs into a typed Arrow batch, using a fixed schema generated from the dataset's specification;
hands the batch to Rust through 7, Go's way of calling C-style code;
runs a query plan that was compiled once and is reused for every batch in a time slice.
Ownership of memory is kept strict. Rust only borrows Go's input buffer during the call. Go copies Rust's output and then tells Rust to free it.
Moving whole columnar batches across the language boundary avoids converting each record individually. One prepared Transformer handled a synthetic 5,000-row batch in about 3 ms, roughly 1.6 million rows per second on a single goroutine (a lightweight Go thread).
Failing safely
Transformers 8. If a transform fails, Bufstream neither delivers the raw logs nor moves past them. Instead it retries with growing delays, reports the error, and eventually disables the job if it stays broken. This means unredacted data never leaks, at the cost of stalled delivery.
The post gives no end-to-end latency or production CPU figures, and it doesn't list which SQL features are supported.
Cloudflare has made Transformers generally available for 1, its service that exports customer logs in near real time. A Transformer is a SQL query that runs on each batch of logs before it leaves Cloudflare. The WHERE clause decides which records are kept. The SELECT clause decides what each exported record looks like. Logpush itself is also now open to all self-serve plans. The post, by Rob Taylor, mostly explains how Cloudflare put user-written SQL into a pipeline that moves more than 110 million records per second.
The problem
Teams export Cloudflare logs to 2 (security information and event management systems), analytics platforms and object storage. Before the logs arrive, teams often want to drop health-check traffic, strip sensitive fields, or reshape records to match a shared schema. The post's example is a team that collects logs from several CDNs (content delivery networks) and adds a static provider field so all the records line up. Today that usually means running a separate processing service. It also means paying to export, ingest and store records the team throws away as soon as they land.
With Transformers, a customer writes a query for a specific dataset, such as http_requests. The query can filter rows and add, rename, remove or nest fields. It can also redact data, for example masking part of a client IP. Output is 3 (newline-delimited JSON, one JSON object per line). Queries can be written and previewed against a sample record in a dashboard tool called Transformer Studio, or managed through the API. Every edit creates a new version.
Why this is hard
4 is Cloudflare's Logpush delivery service. It runs on hundreds of Kubernetes pods. Each pod claims a time slice of data from Buftee, Cloudflare's in-house distributed message broker built on logs. Jobs vary enormously in size: most are small, but a few move millions of records per second (the post calls this a Pareto-like distribution). Any transform layer had to keep delivery reliable, throughput high and resource use predictable, without adding new ways to fail.
The first prototype ran Transformers as WebAssembly code in Cloudflare Workers, the company's isolated serverless compute platform. That put a network call in the export path, and network calls bring timeouts and retries that cut throughput and add latency. Cloudflare dropped the prototype and chose to run transforms in-process, inside Bufstream.
The mechanism
Cloudflare picked 5, a query engine written in Rust that already powers its Cloudflare Basin product. DataFusion is built on 6, a language-neutral columnar memory format. It includes a SQL planner and vectorized execution, meaning it works on whole columns at a time. Bufstream is written in Go, so the real engineering was the boundary between the two languages.
Here is the path a batch takes, step by step:
Cloudflare admits the boundary is not free: batches still have to be serialized, decoded and managed. The point of Arrow IPC is that this work happens once per batch, not once per record.
Numbers
In a benchmark, one prepared Transformer ran against a synthetic 5,000-row http_requests batch from a single goroutine in about 3 ms. That works out to roughly 1.6 million rows per second. A Transformer adds CPU work to its job, so Bufstream runs transform batches in parallel for high-volume jobs. Scheduling, batching and destination controls are the same ones the rest of Logpush already uses.
Failure semantics
Transformers 8. If a transform can't run, Bufstream does not advance the job's delivery offset (its record of how far through the data it has delivered), and it does not send the untransformed batch. Sending raw logs could break a customer's redaction or filtering guarantees, so Bufstream retries instead. To stop one broken job from eating capacity that healthy jobs need, retries back off progressively and errors are reported on the job. Jobs that stay broken for an extended period are disabled automatically. A Transformer Health view shows successes, failures, data volume before and after the transform, and how long transforms take.
What's left open
The benchmark is a single-goroutine, synthetic microbenchmark. The post gives no figures for end-to-end latency, for production CPU overhead, or for what serializing across cgo costs compared with query execution. It doesn't define "extended period" for auto-disable, or say which SQL features are supported. The trade-off is clear, though. Fail-closed delivery guarantees that raw logs never leak, but a bad query stalls delivery until someone fixes it. Also, both input and output are NDJSON: Transformers reshape records but do not convert between formats.
Remember
Cloudflare dropped a Workers prototype because a network call on the export hot path meant timeouts and retries.
Explicit buffer ownership across the Go/Rust boundary sidesteps cross-runtime memory bugs; Go copies output, then Rust frees it.
Fail-closed design means a broken query halts delivery rather than leaking unredacted or unfiltered logs.
Built on 8. QUIC's independent streams avoid TCP's 9. It also sets up connections faster and supports connection migration, which keeps a session alive when a phone switches from Wi-Fi to cellular. Meta's earlier move from HTTP/2 to QUIC lowered request errors, tail latency and video stalls. MoQ also gets any future QUIC feature for free, such as forward error correction (FEC), which sends redundant data so some losses can be repaired without resending.
Fan-out and caching in the protocol. Relays do request coalescing: if a thousand viewers ask for the same object, the relay fetches it once. 10 (content delivery networks) already do this for live streaming. MoQ makes it part of the protocol instead of an add-on.
Latency set per subscriber. Two subscribers to one track can run at different delays at the same time. For example, about 100 ms for someone in an interactive session and several seconds for an archiving service.
Track-aware delivery. Relays can drop, prioritize or replay objects based on metadata. A byte-stream protocol can't do this, because it sees only bytes. Under congestion, a relay could, for instance, drop stale frames instead of queuing them.
Media context. The protocol itself models quality variants, so 11 logic, which switches video quality as bandwidth changes, can use native MoQ constructs.
TCP fallback. When QUIC is blocked, MoQ can run over TCP using a standard mapping: WebTransport over HTTP/2, or the emerging 12 mapping.
14
Translation at the edge. MoQ connections end at dedicated edge servers. There, a translator converts MoQ frames into the RUSH format the existing backend already understands, so nothing downstream had to change. The shim is temporary. Once MoQ is the only ingest path, the backend will read MoQ directly.
Server-side routing. When the system generates a stream URL, it sends broadcasts in the experiment group to MoQ-capable endpoints. Ramping up or rolling back therefore doesn't depend on users updating their apps, and Meta could resize the experiment in minutes.
Columns, not rows. Buftee hands Bufstream compressed batches of Cap'n Proto records (Cap'n Proto is a binary serialization format). For jobs that have a Transformer, Bufstream decompresses the batch, parses the logs and builds a typed Arrow RecordBatch, which is Arrow's unit of columnar data.
A fixed schema. Bufstream does not guess field types from whatever logs happen to be in a batch. It supplies a schema generated from the dataset's specification. DataFusion therefore always sees the same fields and types, and can validate queries against them. The table named in the query's FROM clause picks which schema applies.
Crossing into Rust. The batch is serialized as Arrow IPC, Arrow's standard format for moving batches between processes or languages. It then passes through a C ABI (application binary interface, the low-level calling convention) that Go calls via 7, Go's mechanism for calling C code. On the Rust side, an Arrow StreamReader reads typed batches directly. Nothing is re-parsed row by row from JSON.
Explicit memory ownership. Go lends Rust a pointer to the input buffer only for the length of the call, and Rust does not keep it. Rust owns the output buffer. Go copies the output into Go-managed memory and then tells Rust to free its allocation. Because neither runtime ever shares ownership, a whole class of cross-language memory bugs is avoided.
Plan once, run many times. Saved queries are stored in R2, Cloudflare's object storage, and fetched through a multi-layer cache. Bufstream compiles each query into a DataFusion logical plan, the engine's internal description of the scan, filters, projections and expressions. It does this once per time slice and reuses the plan for every batch in that slice.
Back out. The resulting Arrow output is converted to NDJSON, batched, gzip-compressed and uploaded. Bufstream adjusts how many upload goroutines (lightweight Go threads) it runs, within each destination's concurrency limits.
