Shard keys, pointer heads and safety cases · Morning Gist
Tuesday, October 65 stories · 11 min
Shard keys, pointer heads and safety cases
Today: re-keying shards when one tenant gets too big, routing agent edits through existing write paths, checklists for RL training runs, and two cheaper frontier-class models.
Picked from 63 new articles across 23 sources · arrow keys for other days
Today in 30 seconds
30s left
PlanetScale's Neki post gives three escalating fixes for hot tenants in sharded Postgres, using Slack's switch from workspace- to channel-keyed messages. story 1 ↓
Marc Brooker swapped Qwen3.5-2B's LM head for a 1M-parameter pointer head on one RTX 3090, tying for first among ≤2B models on JevBench. story 2 ↓
Dropbox's Reclaim sends user, scheduler and LLM agent edits through one "Schedule Actions" path and previews them with a pure-function scheduler. story 3 ↓
OpenAI proposes safety cases before frontier RL runs: containment, monitoring, leadership vetoes, fail-closed controls. It's mostly reliability engineering, with no metrics or thresholds. story 4 ↓
Claude Sonnet 5.5 keeps Sonnet 5's price, runs faster, joins the free tier, but inherits Opus 5.5's runaway "max" thinking bug. story 5 ↓
OpenAI says GPT-6.1 Sol nears GPT-6 Astra at a fifth of Astra's API price. Simon Willison checks the claim with his own tests.
1/5PlanetScale· Simeon Griggs· Sep 28
When a tenant outgrows its shard, change the shard key, not the hardware
PlanetScale lays out three escalating fixes for hot tenants in Neki, its sharded Postgres, with Slack's move from workspace-keyed to channel-keyed messages as the example.
Why it matters · Sharding by tenant is the default for most multi-tenant SaaS. This post spells out how that default breaks once you land a whale customer and what each fix costs, which matters before you choose a shard key, not after.
Image: PlanetScale
Sharding by tenant_id gives you an even spread of tenants. Simeon Griggs argues that once a product succeeds, you want an even spread of data instead, and those are different goals. A few huge tenants and new cross-tenant features eventually mean your shard key is wrong, at least for some tables.
The Slack precedent
The post leans on Slack's published history with Vitess, which is built by the same team that makes Neki. Slack first sharded messages and channels by workspace ID and kept the workspace-to-shard mapping in a separate metadata cluster. That rested on the assumption that no single customer would outgrow the biggest database. Enterprise workspaces with more than 100,000 users broke it. Some messages tables got too big for any one shard, and cross-workspace messaging and Enterprise Grid didn't fit the model. Vertical scaling and isolation bought time, but not enough.
The fix was to shard each table by the key that matched how it was accessed: messages by channel ID, others by user ID or workspace ID. Messages were always queried by channel, and channel ID was already in the primary key. Fetching a channel and its messages became a single-shard query, even when authors came from different workspaces. Cross-channel queries became cross-shard queries, but those were admin and batch work, not the critical path. The users table stayed grouped by workspace because "all users in this workspace" is a hot query.
Three fixes, in order of permanence
Scale up. Each Neki shard is its own Postgres cluster, so you can give the hot shard a bigger configuration profile than the rest. Vitess does the same with per-shard cluster sizes.
Marc Brooker builds a 2B calibrated classifier at home, tops JevBench
He replaced Qwen3.5-2B's LM head with a 1M-parameter pointer head, and it ties for first among models of 2B parameters or fewer on a single RTX 3090.
Why it matters · Calibrated, low-latency decision models are a practical building block for agent workflows. Brooker's log shows which changes moved accuracy and calibration and which didn't, with numbers, on hardware an engineer can own.
TypeSafe AI's Jev, a general-purpose calibrated classifier released on September 15, got Marc Brooker to build his own version, Hobson. He capped it at about 2 billion parameters so it would run on his home GPU. On the JevBench public set it ties for first among models at 2B or below, level with decider-2b's v10 entry. It beats Qwen3.5-2B on both accuracy and calibration. On the easy set it scores 100% accuracy with a multi-class Brier score of 0.009. Brier runs from 0 (perfect) to 2 (confidently wrong).
How it works
Brooker starts from Qwen3.5-2B, removes the LM head so the model can no longer generate text, and attaches a pointer head of about 1M parameters. The head scores the hidden state at each option's position against the hidden state at the <answer> position. The rest of the model is fine-tuned with a rank-16 LoRA. He reports better calibration than reading output logits from an LLM of the same size.
His earlier slot head had only 53k parameters. It read only the <answer> hidden state through a linear layer with 24 fixed slots. That capped questions at 24 options, ignored the hidden states of each option, and picked up positional bias. Switching to the pointer head (v7) was the single biggest win.
Training details:
One epoch of cross-entropy against the gold options, with options shuffled on every example.
KL self-distillation toward a frozen copy of the base model to limit forgetting. On tasks that regressed, it distills toward an earlier version of Hobson instead.
115k rows: about 113k from public datasets plus 2k synthetic hard questions. Synthetic examples were generated by Qwen3.5-27B and checked by Qwen3.5-397B.
Reclaim adds AI agents by sending them through its existing scheduler
Dropbox's calendar assistant sends user, scheduler and LLM agent changes through shared "Schedule Actions," then previews them using a scheduler rewritten as a pure function.
Why it matters · This is a concrete pattern for adding LLM agents to a stateful product that already has business logic: one write path for every caller, scoped tools, and a dry-run preview before side effects reach other people.
Image: Dropbox Tech
Reclaim, the calendar assistant Dropbox owns, used to change calendars in two ways: users made direct edits, and a background scheduler moved tasks, focus time and habits around based on preferences and availability. Adding natural-language requests created a third caller, and it doesn't behave the same way every time. The engineers' example is "make time for this tomorrow." An LLM could read that as finding an open block, creating an event or moving an existing one, and some of those choices change other attendees' calendars.
The team did not build a separate AI-only path. They argue that would have duplicated scheduling logic and made behavior drift between interfaces.
The agent platform
The model works out what the user wants. The agent runs the loop around it: it supplies instructions, calendar context and tools, then sends tool results back to the model for the next step. Each request gets only the context and tools relevant to it, not everything at once. Complex requests can hand parts to specialized subagents, and internal checklists and review steps keep the overall request on track.
Reclaim built the agent loop, tools and context gathering in-house and calls model providers directly. The framework it first tried lagged behind provider APIs and had more structure than Reclaim needed. The authors accept the extra code to maintain and say using agents to help write it offsets some of that work. The same tool system serves Model Context Protocol (MCP) in both directions. An MCP client lets Reclaim's agent use outside tools. An MCP server exposes selected Reclaim tools to Claude and ChatGPT, which run their own models and agent loops.
4/5OpenAI· Sep 28
OpenAI proposes safety cases as a precondition for frontier RL training runs
A checklist covering alignment, containment, monitoring, leadership vetoes and fail-closed controls, which OpenAI says it is implementing internally. It includes no metrics or thresholds.
Why it matters · Much of this is ordinary reliability engineering (immutable logs, on-call SLAs, fail-closed defaults, rollback, postmortems) applied to RL training runs. It shows how a frontier lab thinks a training run can be audited before it is allowed to continue.
Image: OpenAI
OpenAI argues that frontier reinforcement learning runs should not continue without structured safety documentation. The goal is a safety case, meaning a structured, evidence-based argument about risk of the kind used in aviation and nuclear power. OpenAI calls this an "aspirational north star." It concedes that AI's emergent complexity makes that level of rigor hard to reach. The post covers training only. It does not cover deployment, and it gives no thresholds, numbers or results.
Three layers of technical safeguards
Alignment is the first line of defense, and it focuses on not rewarding reward hacks. Agents and human reviewers look for broken RL environments that an exploit could score highly on. Graders are tuned to penalize hacking attempts. Classifiers run over traces from earlier runs to check that the graders behave as intended. Alignment evals run during training. They are backtested against past incidents so they aren't overfit to one case, and they are stress-tested by deliberately eliciting worst-case behavior. Monitors track eval awareness ("metagaming") and have blocking thresholds. Graders never see chain-of-thought, so the model has no training pressure to learn to hide its reasoning from CoT monitors.
Containment assumes the model may already be misaligned:
The sandbox and the research infrastructure around it are both hardened.
Frontier checkpoints are used to red-team the containment, including services the model can touch, such as the internet cache.
5/5Simon Willison· Sep 28
Claude Sonnet 5.5: Sonnet 5's price, faster, and now on Claude's free tier
Willison's first look finds Sonnet 5.5 close to Opus 5.5 on some coding tasks, but it shares Opus 5.5's runaway "max" thinking bug.
Why it matters · If you send agentic or coding work to Sonnet, this is an upgrade at the same price that should cost less per task. The "max" effort failure is a real risk if you set effort levels in production.
Sonnet 5.5's pelican-on-a-bicycle SVG at "xhigh" effort: correct frame, pedals and chain; 41 seconds, 5.74 cents.
Anthropic released Claude Sonnet 5.5 on 28 September. Simon Willison covered it in a short link-blog post. It costs the same per token as Sonnet 5, and Anthropic says it "runs 30%+ faster, and costs up to 30% less for most work." The lower cost comes from how the model runs, not from a price cut. The likely reason is that it gets through typical tasks with fewer tokens, though the post doesn't spell this out. Willison notes that it appears to beat Sonnet 5 on every benchmark Anthropic published. He doesn't reproduce the numbers.
What the hands-on testing showed
Willison ran his usual test: asking the model for an SVG of a pelican riding a bicycle, at different thinking-effort levels. Two results matter:
At "max" effort, it failed. The model reasoned for 128,000 tokens, costing $1.28, then hit its output limit and never produced the SVG. Willison says Opus 5.5 has the same bug. Anyone who sets effort levels through the API should note this: the highest setting can use up the whole token budget and return nothing.
At "xhigh" effort, it worked. It returned a good drawing in 41 seconds for 5.74 cents. The bicycle frame was correct, the legs sat either side of it, the feet were on the pedals and the chain was in the right place. The only flaw was an oddly shaped helmet.
Willison also says Sonnet 5.5 looks "almost as good as Opus 5.5 on some coding tasks," including the 3D animation demos that go viral. He gives no numbers for this; it's an impression from his own use.
Isolate the whale. Neki's declarative data topology assigns hashed key ranges (xxhash) to shards. You can carve a narrow range around one tenant's hash, such as 80a3f1–80a3f2, onto a dedicated shard. Resharding copies rows to new shards, so this means resharding every tenant into three fresh shards. You can't reuse the old ones. The catch: one tenant can now own one huge table that itself needs sharding.
Reshard specific tables. Look at query shapes, find tables that are joined on a common column, and shard them together by that column. Most tables, users included, never need resharding.
Reshard copies existing rows, streams new writes while the source keeps serving traffic, then switches reads and writes over without downtime. The post doesn't give numbers on copy time, cutover lag or performance impact.
Griggs contrasts explicit, declarative placement with automatic sharding, where row placement can be unpredictable. He argues that an explicit topology is easier for both humans and agents to reason about. This is a vendor post, and it reads partly as a pitch for Neki. The useful parts are the Slack case and the order of the three fixes.
Remember
Scaling up and isolating a whale tenant only buy time. The real fix is re-keying the specific tables that hurt.
Slack resharded messages by channel ID, which was already in the primary key, so hot reads became single-shard queries.
Neki reshards into new shards only, streaming writes during copy and cutting over without downtime. No performance numbers given.
After training, he fits one temperature per question type (binary, choice, score) on held-out data to minimize log loss. At inference, raw logits are divided by that temperature.
Inference takes one forward pass for a single question and two for any larger number, because the prompt's forward pass is cached. On a 3090, p50 latency is just over 100ms and p95 is under 300ms, driven mostly by prompt length. Brooker estimates the fixed latency floor could fall to about 10ms on the same hardware.
What worked and what didn't
These helped: the new head, a newer and slightly larger base model, and document data checked by a teacher or verifier model. These didn't: templated synthesis, distillation on small or easy problems, an instruct variant of the base model, and adding ShARC and ConditionalQA. Brooker argues that more kinds of problems help, while more of the same data does little. Generalization to unseen tasks is useful but weak, and much harder to improve than in-task accuracy.
He also admits the evaluation was shaky. Moving to Qwen3-4B (v2) looked worse only because the held-out set lacked the right tasks. He has seen the JevBench examples and designed the synthesis process himself, so contamination comes down to his own honesty. Agents wrote every line of code. Next he plans harder synthetic data and reinforcement learning aimed at confidence-threshold decision utility, which can't be differentiated.
Remember
Swapping the LM head for a pointer head that scores each option's hidden state was the biggest single gain.
Diverse, verified data mattered more than volume; templated synthesis and more of the same data barely helped.
Two forward passes handle any number of questions; p50 latency is about 100ms on an RTX 3090.
Every scheduling operation, such as creating or updating an event, changing an RSVP or finding availability, is a Schedule Action. Users, the scheduler and agents all use the same Schedule Action types, with the same validation and commit process. That matters because one move can shift flexible events and change other people's availability. With a single implementation, engineers change an operation in one place, not three.
Preview Mode shows a temporary copy of the calendar so users can review proposed changes, including effects on shared events, before any updates go to attendees. To keep it fast, the team rewrote the automated scheduler as a pure function: it computes a proposed schedule without saving anything, and it reruns whenever the user adjusts the preview. Calendar and attendee data for busy schedules is cached in Redis. When an event moves inside the preview, attendee availability updates so the next calculation reflects the proposed state.
The post gives no latency, accuracy or adoption numbers.
Remember
All changes, whether from users, the scheduler or LLM agents, go through shared Schedule Actions with one validation and commit process.
Reclaim built its own agent loop instead of using a framework, which lagged provider APIs. MCP works in both directions.
Preview Mode depends on rewriting the scheduler as a pure function, with Redis-cached calendar data to keep recalculation fast.
High-bandwidth communication between samples is limited to approved channels, and easily found side channels are closed.
Transcripts are kept in write-once storage so incidents can be investigated.
Monitoring has to show high recall on held-out past incidents. Its evals are refreshed with new-risk datapoints so precision and recall measurements don't go stale. It also has to pass monitorability and monitor-evasion evals. Priority alerts have an SLA. For example, during the day an alert pages security on-call and the run owner. At night, a run whose alert goes unacknowledged is paused automatically.
Operational and incident practices
Someone from another team writes a dissent (a pre-mortem) against each safety case.
Several senior leaders each hold a veto. The examples given are the research VP, the Head of Safety and the Chief Scientist.
The responsible leader is accountable for the run, including in performance reviews.
Monitoring and auto-pause fail closed: a run can't start without them, and they can't be disabled from inside training.
Every downstream use of a model, such as data generation or grading, is tracked so a misaligned model's outputs can be rolled back.
Residual risks are listed explicitly.
Misalignment severity levels are defined, with an on-call that can page the CEO.
For incidents, OpenAI recommends:
root-causing through ablations and resampling
cultural as well as technical postmortems
regression evals built from incidents, with detection methods developed without hill-climbing on incident transcripts
public disclosure once the investigation ends
OpenAI says these practices are being implemented and will change "over the coming weeks."
Remember
Graders never see chain-of-thought, so models get no training pressure to evade CoT monitors.
Monitoring and auto-pause fail closed, and unacknowledged alerts at night pause runs automatically.
This is a policy proposal with no reported metrics, thresholds or evidence that it works.
Willison argues that the most interesting part of the release is distribution. Sonnet 5.5 now runs the free tier of claude.ai. ChatGPT's free tier uses Luna 5.6, so Willison says Anthropic now offers a "much more capable" free product. To test it, he asked the free tier for a WebGL page showing a 3D pelican riding a bicycle. He called the result "a solid effort."
Anthropic says Haiku 5.5 will arrive "in the coming weeks." Willison hopes it can compete on price with OpenAI's GPT-6 Luna. That suggests he thinks Anthropic's cheapest model has been losing out on price.
The post is a quick first look, not an evaluation. It includes no benchmark tables, no latency figures beyond one run, and no head-to-head coding tests.
Remember
Same price as Sonnet 5; Anthropic claims 30%+ faster and up to 30% cheaper per task on most work.
"max" effort used 128K tokens ($1.28) and produced nothing; "xhigh" succeeded in 41s for 5.74¢.
Sonnet 5.5 now runs claude.ai's free tier; Willison calls it much stronger than ChatGPT's free Luna 5.6.
