Exact Counts, Lazy Heights, Unpaused Isolates

The news

Keep in mind what kind of post this is. A vendor is benchmarking against a competitor, using a demo someone else built and sized, and it ends in a sales pitch. It gives no internals for TIN, no methodology, and no repeated measurements. The interesting part is less the speed claim than the list of things the app could do once queries were cheap.

What caught Reynolds's eye: a tilde

The original demo shows "~3,400 results" for the query show hn, and the query takes more than 3.6 seconds. Reynolds asks two questions. Why can't the database give an exact count? And why does ranking only 3,400 documents take seconds?

The answer is the actual count: exactly 213,447 matches. So Lakebase was ranking a match set about 60 times larger than the number it displayed. The post doesn't say how the original app produced its estimate. A common pattern is to read the query planner's row estimate, because an exact COUNT(*) over a large match set is expensive in stock Postgres. Treat that as a plausible explanation, not something the article states. Reynolds says TIN is "especially well optimized" for COUNT(*) and returns exact counts for large, multi-term queries in a few milliseconds.

Feature 1: no debouncing

Feature 2: autocomplete through wildcards

TIN supports prefix wildcards. The app treats an unfinished final word as a prefix, so typing planetscale datab runs the query planetscale datab*. That matches "database". It would also match oddities like "Databall" and "DATAbEEF", but those don't co-occur with "planetscale". Results appear before the user finishes the word, and users who can't spell a word get help.

Feature 3: fuzzy matching with a fallback strategy

1. Run the query exactly as typed.
2. If it returns too few results, run a separate COUNT(*) for each term on its own.
3. Allow an edit distance of 2 on the rarest term and run the query again.
4. If results are still too thin, fuzz the next-rarest term, and so on until every term is fuzzed.

A worked example from the post: planetscale databse. The misspelling "databse" appears only ten times in all of Hacker News, so it is the rarest term. The app rewrites the query as planetscale databse~2, which matches "database". The UI marks which words were fuzzed.

This approach makes several extra count queries for one user search. That only works if counts cost milliseconds, which ties the feature back to the counting claim.

Most setups, Lakebase's included, are configured to drop stop words. A query like "to be or not to be" or "The Who" then reduces to nothing, and the Lakebase app shows no results. TIN indexes every word. While Reynolds typed, the query to b* matched 1,054,718 documents, and TIN ranked the top 30 in 11 ms. The post doesn't report what indexing every word costs in index size or write throughput. That is the usual reason to drop stop words.

Cost

A Neon CU (compute unit) is a bundle of CPU and memory. Here 32 CU means 32 cores and 128 GB of RAM. The Lakebase figure is 32 CU × 730 hours × $0.222, with autosuspend turned off so the instance never sleeps. The TIN price covers a three-node high-availability (HA) cluster with local NVMe (fast solid-state) disks. That is under one-eighth the price, with a sixteenth of the cores and an eighth of the RAM per node.

How much to trust it

The direction of the result is believable. Stock Postgres full-text search is known to slow down when it ranks or counts very large match sets, and the gap here is large: seconds against milliseconds. Some caveats still apply:

• The baseline was chosen by someone else. Rishi sized the Neon instance and turned off sleep. A smaller or autoscaling configuration would change the price comparison. The post doesn't say whether the Lakebase side could have been tuned, for example with different ranking queries or indexes.
• No methodology. The timings come from single screen recordings, with no percentiles, cold or warm cache details, or concurrency.
• Missing costs. The post covers no index size, write or ingest speed, or rebuild time.
• TIN is a black box here. The post doesn't explain how TIN gets fast counts or fuzzy matching. An earlier PlanetScale post is cited for raw speed but not summarized.

The engineering point holds regardless of vendor. If counts and fuzzy lookups take milliseconds, the application can afford to run extra queries per user action, and that opens up features a slower backend would rule out.

Why code is easy and comments are not

General-purpose virtualizers start with estimated heights and correct them as rows get measured. When every row is a line of code at a known font size, though, the whole table can be computed up front and never changes. GitHub calls this the "all heights known before paint" contract. The diff surface is built on it:

No per-frame work grows with the total row count, and the team kept all of it.

Comments break the contract. A thread's height depends on how its markdown wraps at the current width, collapsible <details> sections, a reply box that grows as you type, suggested-change diffs, reactions, and images that change height when they finish loading. None of that is known until render time, and it keeps changing afterward. Reserving a fixed estimated slot fails at the extremes. Most comments get too much space and leave whitespace gaps. Long ones get too little and get clipped or grow a nested scrollbar. Writing the measured height back into the shared offset table moves everything below it while the user is scrolling, which shows up as a visible jump.

So comments got a weaker promise: heights are bounded and measured lazily, and corrections are small and anchored to whatever the user is looking at.

Two geometries

The key step was to split the height into two independent parts:

total = exact code height + Σ dynamic block heights + scroll padding

Code geometry stays deterministic and is never rebuilt when a comment resizes. Each dynamic block (a thread, draft or reply composer) has three properties:

A block's effective height is the first of these that applies: a valid measurement, then a cached height whose fingerprint and width bucket still match, then the estimate. Block heights sit in their own index. Their count grows with comments, not rows, so a few thousand is manageable, provided first paint doesn't measure them all.

The measurement scheduler

The first design gave each block its own ResizeObserver that wrote measured heights back into the layout. The team rejected it during performance hardening. An observer that changes the layout of the element it watches can retrigger itself, and the cost grows with every mounted block.

What shipped is a single measurement pass that waits for idle time and for scrolling to stop:

• Off the hot path. The pass runs when the visible range settles, never once per frame, and waits while a scroll is in progress.
• Scoped to the viewport. Only blocks within roughly 2,400px of the screen qualify, so the work is proportional to the viewport. Distant blocks keep their estimates.
• On-screen reads win. The pass reads all mounted candidates in one batch: one reflow, with no writes in between. It never skips a mounted block in favor of an estimate. Breaking this rule caused the worst bug the team hit: blank strips under comments that had been filtered out of measurement and left on estimates that were too tall.
• A bounded off-screen fallback. A nearby block that hasn't mounted gets at most one off-screen render. Blocks taller than the viewport skip even that, because their extra reserved space hides below the fold.
• Observers only flag. Each mounted block's ResizeObserver just marks the block for the next pass to re-read. It disconnects on unmount.

There is one deliberate exception. When a user expands a section or opens a reply, waiting for the next pass showed a one-frame mismatch: the comment grew but the code below it hadn't moved yet. For mounted, on-screen blocks, the observer now applies the correction before paint. Two safeguards keep the loop closed: at most one synchronous commit per frame, and none during an active scroll.

Scroll anchoring

Corrections work by identity, not by pixel:

1. Record the row or block the user is anchored to, plus the offset within it.
2. Apply the height changes.
3. Find that anchor's new position.
4. Scroll so it stays put on screen.

A few rules sit on top. A block above the viewport that changes height gets compensated. Content filling in below the viewport is left alone. A section the user toggles themselves lets the content below flow down naturally. Wheel momentum is never fought.

That last rule caused a subtle bug. The "user is scrolling" guard used the timestamp of the last scroll, and scrolls the app made itself refreshed that timestamp too. Toggling the file-tree sidebar changed the pane's width. With line wrapping on, the whole coordinate space shifted and the surface emitted a small scroll of its own. The guard then skipped the correction, and the user's file drifted off screen. Gimeno's lesson: any "is the user interacting?" check must be one your own side effects can't satisfy.

The data pipeline

Three habits support the UI:

• Structure before content. The diff streams so the file tree and metadata appear first, and all review threads are resolved up front.
• Deferred per-item work. Syntax highlighting runs off the main thread, so rows show as plain text and get colored later. Large markdown bodies aren't built until they approach the viewport.
• A small cache. Diff documents are still released on navigation to save memory. But the metadata persisted, so the header and file tree reappeared instantly around an empty diff, which looked broken. The team now keeps the last few diffs in memory and refreshes stale ones in the background.

Finding the bugs

Bugs like "whitespace below some comments, sometimes, and it heals if you scroll back" were found by tooling rather than by eye. The surface carries permanent structured probes that check its own invariants:

• how many rows and blocks are mounted
• whether measurement coalesces into one commit per frame, and how long that frame takes
• how large the scroll corrections are
• whether any block appeared after scrolling began (this must be zero)
• whether observers tear down on unmount

These checks run as budgets in an end-to-end CI test against a synthetic huge-PR fixture. A headless lane runs JSON-described flows against a mock server and ranks bottlenecks using render counts, the browser's performance timeline and a requestAnimationFrame jank sampler. An autopilot drives the real desktop app, cold and warm, through toggles, sweeps and resizes. It logs a health signal with each sample: no unfilled gaps, no blank blocks, real thread content mounted. Once a bug is understood, the temporary probes come out and the invariant is pinned in a test.

What the article leaves open

The post reports no before-and-after numbers: no open time, frame times, correction sizes or memory use. The only claim is that the extreme pull request now "behaves like a normally sized pull request." It doesn't describe the height estimator, the width bucket size or how many diffs the cache holds. It also doesn't compare its approach with built-in browser features such as CSS overflow-anchor or content-visibility. These features probably don't fit a recycled, imperative surface, but the post doesn't say so.

The problem: the descriptor arrived too late

The article names a second gap. IDEs do so much for Java developers that Java never grew small command-line tools for formatting, symbol lookup or browsing documentation. Thomas says this shows up when coding agents, which work through a terminal, try to find Java dependencies, docs and sources.

Making the descriptor complete

ja puts versions and metadata into the descriptor itself, using comments and Javadoc-style tags the compiler ignores:

/**
 * @mainClass com.example.application.Main
 */
module com.example.application {
 requires com.example.framework; // @1.2.3
}

The // @1.2.3 comment pins the dependency version next to its requires line. The @mainClass tag sets the entry point. Commands follow the style of other languages' package managers, for example ja require [email protected].

Small tools that compose

ja itself only handles command-line ergonomics and orchestration. Each feature is a separate tool you can use without ja:

• jig resolves module versions, compiles and assembles. It prints standard module-system arguments that other tools can consume. It also connects to Maven repositories, with a standalone module proxy and publishing commands.
• jfmt formats code using the old Code Conventions for the Java Programming Language, updated for modern Java. It targets the whitespace, import-ordering and fully qualified name problems the team says are common in machine-written code.
• jist is a grep-style search over compiled class files and their sources. It needs no indexing, Language Server Protocol (LSP) server or other protocol server, and it works with other build tools through an argument-file contract.
• jdocserver serves API documentation locally.

All of them implement the JDK's Tool or ToolProvider interfaces. These are the standard mechanism for discovering a tool and running it inside an existing JVM process, so the tools can be installed into the JDK next to javac and jar. ja uses OptionChecker to ask each tool which module-system options it accepts, then fills in arguments such as --module-path itself. That means the same project setup works with standard JDK tools: jdeps (dependency analysis), jlink (custom runtime images) and jshell (the REPL).

Finding modules in a repository that doesn't know about them

Maven Central identifies artifacts by groupId:artifactId:version, not by module name. In the team's survey of the 1,000 most popular Maven Central artifacts:

The module system also doesn't separate a namespace from a module name. So a tool that starts from requires com.foo.bar has to work out which artifact to download.

Integrity by default

Integrity by Default is the OpenJDK effort to restrict deep reflection, native access and, under the "Preparing to Make Final Mean Final" work, changing final fields at runtime. Each opt-out is a launcher flag naming the modules allowed to break the rule. On the class path that name is usually ALL-UNNAMED. Thomas argues that this hides which library is building up the technical debt.

ja lets a library state what access it needs:

/** @enableFinalFieldMutation com.example.framework */
module com.example.framework { }

The library cannot grant itself that access. The application must repeat the tag to authorize it, or add it in one step with ja require [email protected] --enable-final-field-mutation com.example.framework. Without that authorization, dependency resolution fails with an "unsatisfied access requirement". The problem appears at build time, not at runtime. @enableNativeAccess works the same way, and qualified exports and opens are supported too.

Two other integrity measures:

• Lockfile-style hashes. Hashes of resolved binary dependencies are stored in module-info.hash. Later resolutions check them and reject any artifact that has changed.
• Annotation processing as code generation. Annotation processors run as an explicit step, and their output is saved next to the module's source. Generated code shows up in code review, and a module can be built without running generator code. The team describes this as going further than recent JDK hardening of annotation processors.

What's left open

PlanetScale has released TIN ("Text INdex"), a full-text search extension for Postgres. It is generally available on the company's Postgres and Neki databases; the post doesn't describe Neki. You create it like any index (CREATE INDEX ... USING tin(col)) and query it with a new operator, WHERE col ==> 'some words'. Ranked queries add ORDER BY tin.score(ctid) DESC LIMIT 10.

The post comes from Eric Ridge and is a vendor announcement, so treat the benchmarks as vendor numbers. What makes it worth reading is the design argument: one choice of document identifier makes nearly every part of the index cheaper.

The bar PlanetScale set

The team says a good Postgres text index must support all of the following:

• Boolean, phrase and span queries
• Fuzzy, wildcard and regex term matching
• Case and accent folding
• Exact COUNT(*) and BM25-ranked top-k results

Benchmarks

The published results use an 85 GB Stack Exchange dump with 150 million documents. There is no standard query log for that corpus, so the team sampled substrings of 2–15 terms. Each was run as a conjunction (all words), a disjunction (any word) and a phrase, for 3,762 queries in total.

Postgres 18.6 ran in a container capped at 8 vCPUs and 32 GB RAM on an AWS i7i.8xlarge with local NVMe. The cap was deliberately small, so the indexes would not fit entirely in memory. The load generator was a fork of ParadeDB's own benchmarker, run on the same machine.

Builds. TIN built in 8m10s within 32 GB, producing a 50.7 GB index. ParadeDB needed 64 GB of RAM and 19m20s (52.1 GB index). pg_textsearch needed 128 GB and 26m49s (41.5 GB). GIN took 2h09m with 64 GB and produced the smallest index, 28.0 GB.

Queries (QPS = queries per second; p99 = 99th-percentile latency):

GIN ran out of memory on the disjunction workloads. pg_textsearch supports only ranked disjunction queries, so it could not run phrase, conjunction or count workloads. Only TIN and ParadeDB finished every test.

Most search engines, Lucene among them, split the index into segments and number documents 1..n inside each one. Dense sequential IDs compress well with delta encoding (storing gaps between IDs) and bit-packing. But Postgres speaks only in ctids. An index extension has to return them, and bitmap scans and every built-in index use them. So engines with their own IDs keep a separate map back to ctids. Per the post, ParadeDB and pg_textsearch both do this, so a search matching 10 million rows means 10 million lookups.

TIN uses the 48-bit ctid itself as the posting: 32 bits of page number and 16 bits of slot offset. That avoids the mapping step, but sparse 48-bit numbers compress badly with standard techniques. TIN instead uses two-level bitmaps:

Across a corpus, common terms come out near 1 bit per posting, medium-frequency terms around 7, and rare terms near 25, against a raw 48. Terms that appear only once aren't stored as bitmaps at all.

Worked example. Take the AND rareword. "the" appears on almost every page; suppose "rareword" appears only on pages 190 and 5,000 (an illustrative case). TIN ANDs the two page bitmaps 256 pages at a time. Only pages 190 and 5,000 survive, so only their offset bitmaps get decoded. For a count of the OR rareword, if the page bitmaps share no bits, the answer is just the sum of the two exact posting counts stored in metadata. No postings are read.

AND and OR become single vector instructions, and counts use the hardware POPCNT (count set bits) instruction. To return rows, the position of each set bit is the ctid. Results also come out in heap order, so Postgres reads the matching rows mostly sequentially.

When a query returns columns, Postgres must fetch each matching row anyway and reports whether it is visible to the query's snapshot. TIN skips the invisible ones.

Deletes go into a per-segment liveness bitmap. When VACUUM removes dead tuples, TIN clears their bits and flags the affected page groups. Queries touching those groups AND their offset bitmaps against the liveness bitmap.

Merging without renumbering

New data lands in mutable segments, which are slower to search. A background worker later converts them into immutable segments and merges small ones into larger ones. In engines with per-segment IDs, merging means renumbering, then recompressing and rewriting everything; the post says that can approach double the storage temporarily. TIN's ctid postings mean the same thing in every segment, so many bitmaps simply change ownership on disk with no copy. That cuts write amplification, meaning extra bytes written per byte of data.

What to weigh

These are vendor benchmarks: a synthetic query set, one machine, and competitors at near-default settings, though the setup is spelled out so others can reproduce it. The speed relies on AVX2 or newer x86 CPUs, and the post says nothing about ARM. TIN's index is about 80% larger than GIN's.

Some questions go unanswered. Can a ctid freed by VACUUM and then reused by a new row ever collide with stale postings? How do HOT updates interact with the index? HOT updates are Postgres's in-page updates that skip index changes when no indexed column changed. How quickly do mutable segments get promoted under sustained writes? The post explains the mechanisms but publishes no data on these.

The problem: logs say that, not where

Aggregate metrics can tell you a Worker is burning CPU or running out of memory. They can't tell you which function is responsible. Until now, Workers could be profiled only locally through Chrome DevTools, and a local run doesn't see production's traffic volume or request mix. Cloudflare's position is blunt: the best way to understand an application is to profile it in production.

Sorting the table view by samples surfaced genericR2JsonReplacer. It took over 5% of CPU and called itself recursively. A JSON replacer is a callback passed to JSON.stringify, and JSON.stringify already calls it on every node of the object tree. This replacer also walked the tree itself, so the work was duplicated. A value five levels deep was processed five times. As an illustration: for a node at depth d, the outer traversal reaches it once and every ancestor's own walk reaches it again, so the cost grows with depth rather than staying constant per node. Removing the extra walk made the function 2.7x faster.

The same profile showed a metrics call made twice in the same code path. One call alone was about 1% of CPU, so storing the first result in a variable saved a noticeable amount. The post doesn't give a combined end-to-end CPU figure for the Worker.

Case 2: telemetry that was "disabled"

The P999 now sits about 10 MB under the limit, and the "Exceeded Memory" error graph dropped to match.

How a profile request finds its target

The client answers some of these. The request URL names both the script and the version. The system deliberately does not start a fresh isolate to profile, because the point is to watch real production execution. The catch is that a Worker with little traffic may be hard to profile at all, and you need to choose a deployed version that is actually serving requests.

Regular Workers are stateless and interchangeable, so the system picks a running isolate at random. Durable Objects have names. You can name the object you want, and the runtime routes the request to the metal that owns that actor and profiles its exact isolate.

Profiling without stopping the program

1. Acquire the isolate and its lock.
2. Create a V8 CPU profiler and start sampling at a 1 ms interval.
3. Release the lock so requests keep running.
4. Wait for the requested duration.
5. Reacquire the lock and stop the profiler.
6. Release the lock, then serialize the result outside the critical section.

Serializing after the lock is released keeps the potentially slow output step from blocking request handling.

Limits and what's next

The post names two limits. First, profiling is on demand: someone has to start it, so a short burst of bad behavior can be missed. Second, the memory profiler only sees allocations made during the profiling window, so memory allocated at startup doesn't appear. Cloudflare says it is building continuous profiling, which would collect samples automatically and make rare events easier to catch. The post gives no figures on profiler overhead, and no date or sampling rate for the continuous version.