Fast Search, Slow Search, Cheaper Search

Stemming changes what gets written into the index, so existing indexes have to be rebuilt to use it. You update the extension, build a new index with CREATE INDEX CONCURRENTLY ... USING tin(col) WITH (stemmer = 'en'), then drop the old one concurrently. The CONCURRENTLY option lets Postgres build or drop an index without blocking writes to the table.

A typical ranked TIN query filters with TIN's ==> match operator and sorts by tin.score(ctid) DESC LIMIT 10. ctid is Postgres's physical row address: a page number plus a slot within that page. Some users want ties broken in a fixed order, for example ORDER BY tin.score(ctid) DESC, created_at DESC, id.

The plain query used a block-max algorithm. This is a standard technique in search engines, described here from general knowledge. Each word's posting list (its list of matching documents) is split into blocks, and each block stores the highest score any document in it could reach. While collecting the top 10, the engine keeps track of the 10th-best score found so far. It skips any block whose maximum can't beat that score, so most matching documents are never scored.

In the GA (general availability, first public) release, adding a tiebreaker column turned that optimization off. TIN had to find and score every matching document. v1.0.6 uses the block-max plan for both kinds of query. The article doesn't say how heap columns such as created_at are folded in when scores tie.

Why TIN skips common words in scoring

The biggest performance lever, and the one ParadeDB had criticized, is a default setting. By default, TIN leaves out of the BM25 score any query term that appears in more than 10% of documents. Such terms still count for matching: a conjunction (match all words) still requires them, and a phrase query still checks their position. They just don't add to the score. Reynolds's reasoning is that BM25 weights each term by its rarity, so very common terms change scores very little. They are also expensive, because their posting lists are the longest.

Here is an illustrative example (not from the article). Take the query how to rebase. The words how and to probably appear in more than 10% of Stack Exchange posts. TIN would still require them to match, but would rank results by rebase alone. That saves walking two huge posting lists, at the cost of slightly different scores.

You can change this per query. tin.full_score(ctid) scores every term. tin.score(ctid, dense_ratio => F) sets a different cutoff. Reynolds compares the approach with traditional stop-word lists, which drop common words from the index entirely so they can't be searched at all. Whether the default still counts as "BM25" he calls a semantic question. The benchmarks report both modes: TIN (default) and TIN_FULL (all terms scored).

Benchmark setup

The setup matches ParadeDB's methodology:

• Data: 85 GB of Stack Exchange questions and answers, 150 million documents.
• Queries: 1,254 synthetic queries, each run three ways: disjunction (match any word), conjunction (match all words) and phrase (all words, in order).
• Ranking: top 10 by BM25 score.
• Hardware: an i7i.8xlarge EC2 instance (5th-generation Intel Xeon). The database container has 8 vCPUs, 64 GB RAM, shared_buffers of 24 GB, and CPUs pinned to cut down migration noise. The benchmarker runs on the same host.
• ARM comparison: i8g instances (Graviton4).
• Versions: ParadeDB 0.26.0 final, with its query-type-specific operators.

The table never changes, the index is built in one pass and freshly vacuumed. The authors call this "basically a best-case scenario". The only stress is that the index is about twice the size of the buffer cache.

Results (x86-64)

QPS is queries per second. p99 is the 99th-percentile latency. On the mixed workload the old v1.0.2 managed 110 QPS (TIN) and 36 QPS (TIN_FULL) on x86. That makes v1.0.6 about 3.8x and 5x faster. On ARM the gain is smaller, from 135 to 265 QPS. A "MB/query" column, which the article doesn't define, falls from 64 to 27 for default TIN.

PlanetScale's summary: with default scoring, TIN has 3.1–5.0x the throughput and 2.7–4.4x lower p99 than ParadeDB on x86. With full scoring the lead shrinks to 1.3–3.0x throughput. The weakest case is full-scored disjunctions, at 85 vs 65 QPS.

Trade-offs and open questions

PlanetScale says ParadeDB 0.26.0's speedups grew its index from 52.1 GB to 67.3 GB (+29%) and require a rebuild. TIN v1.0.6 keeps the same on-disk format, so it is a drop-in upgrade. Stemming, though, does need a rebuild.

Some points to keep in mind:

• This is a vendor benchmarking a competitor. The headline multiples depend on TIN's default approximate scoring.
• The read-only setup doesn't test concurrent writes. PlanetScale says TIN is "dramatically faster" there, because ctid-based document IDs make counting and segment merging cheaper than the sequential ordinal IDs that ParadeDB uses. That claim isn't measured or explained here.
• The post doesn't name the specific optimizations behind the 2–4x gains, beyond block-max and SIMD. It also doesn't explain the ARM regression in p99 for some configurations.

Majors thinks these problems share a root. If leaders take "treating people well" to mean "nobody feels bad," they will assume it costs performance. If people link learning with pleasant feelings rather than discomfort, they probably aren't learning much. And because humans tend to see people where there are really systems, we look for fairness in the wrong places.

Before 2015: the brilliant jerk

DORA and Aristotle started from first principles, asking what leads to the best outcomes. Their answer, which Majors says has grown stronger with each report, is that individual skill matters but delivery and impact happen at the level of the team.

How the concept drifted

The academic definition of psychological safety is a team's shared belief that interpersonal risk-taking is safe. Majors stresses that the research was always about excellence, and that researchers insisted it be paired with accountability. Most of that nuance was lost as the idea spread. Its popular meaning moved toward "I should feel safe at work" and then "I should never have to feel bad at work."

Her examples of people saying their psychological safety had been damaged: hearing bad news, layoffs, a promotion that didn't come, reorgs, a manager answering Slack after 6 pm, and constructive feedback on their work.

Her correction is about cause and effect. Psychological safety matters because it enables learning, and learning leads to performance. Feeling safe is a means, not the goal. She adds a limit the pop version leaves out. The effect is strongest in work that is uncertain, interdependent or changing, where people have to surface problems and adapt all the time. In routine, stable work it matters much less.

"If your team is happy, you're doing a good job"

She now rejects the premise. Management is about business success first, and then, ideally, good outcomes for everyone involved. She quotes her own earlier ranking: the most effective leaders are kind people who are skilled operators. Next come unkind people who are skilled operators. Everyone else comes after that.

The ice-cream test

Her argument against happiness as the metric is a thought experiment. If happiness were the goal, she could give everyone six months off, triple their salaries, and make their job taste-testing ice cream. Would that work?

The line she draws: people's feelings deserve acknowledgment and respect, but they are not "a business outcome we should manage to." A leader who can't stand disappointing their reports won't be effective. Saying no and pushing people is part of the job, and how people react is up to them.

Learning is uncomfortable, and that builds trust

Learning means introspection, which is humbling, and stretching your limits, which is uncomfortable. Admitting in front of peers that you don't know something is vulnerable. That, Majors argues, is why it builds trust. She cites Cat's observation that even hostile teams soften once they start asking "what did we learn together?"

She also warns against learning theater. Pulling ten or twenty people into a discussion might produce the best hour of the month, or it might start endless Slack arguments. You can't tell from the calendar invite, and often you can't tell afterward either. So some leaders underinvest out of skepticism and others overinvest and get nothing back. Her advice is to treat learning like any other investment: form a thesis, test it, invest, and measure whether you were right. Expect slow returns. The essay does not say what to measure.

People get credit for what structures do

Her last point is about attribution. People see intent and faces everywhere, "faces onto grilled cheese sandwiches." So we credit the manager who gave us a raise "because we deserved it," not the leaders who ran the business well enough to fund raises. Majors thinks expecting managers to both do the job and make us feel good is an impossibly high bar. Outcomes come from the work of hundreds or thousands of people, yet we "give individuals too much credit and too much blame, and let the system off the hook."

The essay ends without resolving this. It hands Cat a question: when you measure fairness, do you look at people or at systems, and what exactly do you look at? Readers wanting concrete metrics for team health or learning won't find them here. Majors leaves that for later in the exchange.

The problem: testing against a dependency you can't run

This is a common situation. Production depends on a component, such as a managed service or a proprietary engine, that you can't start inside a test container. The usual workarounds are weak. You can mock the search calls, and then your tests check nothing about query semantics. You can skip search tests entirely. Or you can point tests at a shared remote instance, which brings flakiness, cost and shared state. A search extension is a particularly bad candidate for mocking, because its behaviour is complex. Tokenization decides what counts as a word, query syntax can be intricate, relevance scores determine result order, and results must respect transaction visibility.

PlanetScale's answer is a reference implementation. Lead is a second real implementation of the same interface that gives up performance to be simple and portable.

What Lead keeps

The post lists the parity points:

The post says search results are correct "even for complicated SQL and TINQL queries."

What it drops, and why that's the design

The numbers

The post gives a few measurements:

The War and Peace figure is labelled "unofficial benchmarking." My own rough arithmetic from these figures: 3.4 MB in 170 ms is about 20 MB/s, and 8 GB in 240 s is about 33 MB/s. Both suggest throughput that stays roughly linear in data size, which is what a scan would produce. The guidance follows from this: keep CI tables under a few megabytes and query times stay in the low hundreds of milliseconds.

Trade-offs

The split the post recommends is TIN for production and Lead for tests, CI, development and staging. A few consequences are worth thinking through, though the article doesn't discuss them:

• Performance bugs go uncaught. A query that is slow on TIN, or a plan that behaves differently at scale, won't show up against a scanner that is uniformly slow. Load testing still needs TIN.
• Parity is a promise you have to keep. Two implementations can drift apart. PlanetScale commits to porting tokenizers and asks users to file tickets or post on Discord when Lead differs from TIN. How parity is verified, for example by running a shared conformance suite against both, isn't described.
• Staging on Lead only works if staging data stays small. Many teams' staging environments hold production-sized copies, and there Lead's scan costs would hurt.

The pattern is familiar from local emulators for cloud services, where a functional stand-in lets you test without the production engine. Lead's name is a joke along these lines: tin and lead are both metals, and lead is the heavy one.

What's open

The post leaves several questions unanswered: why TIN itself can't run in CI, what license Lead uses, how differences between the two will be detected, and whether Lead's results and scores match TIN's exactly or only closely. Lead can be downloaded from a public repository now.

• compressing vectors (quantization)
• serving indexes from SSD instead of RAM
• supporting queries where each item has several vectors

The problem

Quantization: smaller vectors

Both methods turn float vectors into integers.

An illustrative example, not from the article: a 256-dimension float32 vector takes 1,024 bytes. With 8-bit SQ it takes 256 bytes. With PQ using 32 subvectors and 256 centroids each, every subvector becomes a one-byte ID, so the vector takes 32 bytes. PQ compresses much harder, but it keeps only the nearest centroid, so it loses more detail.

Pinterest benchmarked both on 100 million GraphSage embeddings. GraphSage is a graph neural network method that produces vectors for nodes in a graph.

IVF (inverted file) is an index that groups vectors into clusters. HNSW shrinks less than IVF, probably because the graph's neighbor links stay uncompressed. That is our inference; the article doesn't say. Because the trade-off was clear, the team ran an online A/B test for each use case to pick a quantizer, requiring negligible impact on engagement. Across major use cases, index memory fell by more than 50% and serving costs by 20–30%.

SSD serving: index out of RAM

NVMe SSDs cost roughly an order of magnitude less per gigabyte than RAM. But an access takes microseconds rather than nanoseconds, so a query that makes many random reads gets slow. To work from SSD, an algorithm has to bound the reads each query makes. Pinterest compared two such algorithms on a 100M-vector corpus from a Search use case:

Pinterest made one change to SPANN: it applied PQ to the vectors stored on disk but kept the in-memory centroids at full precision. That makes posting lists smaller and faster to read and scan. Precise centroids still route each query to the right clusters. Results:

• SPANN+PQ ran 4.5x faster than plain SPANN.
• It delivered more than 3x the queries per second (QPS) of DiskANN at one-third the latency.
• Recall dropped about 5%.

In Manas, cluster centroids are chosen with SPANN's hierarchical balanced clustering, which keeps clusters close in size. Equal-length posting lists mean no query reads an unusually large one, which keeps tail latency down. The centroids are themselves indexed with HNSW, which suits a small set of vectors held in memory.

Multi-embedding retrieval

To support this, Manas gained a new query type. The query parser now handles multiple query embeddings, and Manas runs several ANN searches at once for a single query. A pilot with one product team is in progress. The article doesn't explain how results from the parallel searches are merged and rescored. It also doesn't say how much extra storage per-token document vectors need, which is a known cost of ColBERT-style systems.

What's open

The article reports relative numbers only. It gives no absolute latencies, no p99 figures, no recall@k definition, and no details of how DiskANN was tuned for the comparison. It doesn't say whether "5% recall drop" means percentage points.

Freshness is a gap. Manas promises updates within seconds, yet SPANN's posting lists are built offline. The roadmap names SPFresh, a follow-up design for updating SPANN indexes in place, but doesn't say how updates work today. The plan also includes first-class ColBERT-style retrieval and GPU-based systems named SilverTorch and TIGER, which the article doesn't describe.