Use CasesGuidesMethodology
Guides/How the pipeline works & the market/Search A/B
How the pipeline works & the market

Quoted vs tokenized Reddit search: an A/B test

Searching Reddit for a multi-word phrase: do you want the exact phrase, or any post containing the words? We measured which one actually helps.

01The noise problem

Search Reddit for “dating communication skills” without quotes and the top result is a drama-aggregator subreddit — none of whose posts contain the phrase. Loose, tokenized matching treats your phrase as a bag of words and pulls in anything vaguely adjacent.

Exact-phrase (“quoted”) search fixes the precision but risks the opposite failure: too few results to be useful. We tested whether that risk is real.

02The test

We ran ten keywords — a mix of one-, two-, and three-word phrases across two domains — through the same Reddit search endpoint the pipeline uses, quoted and unquoted, and counted results. The bar: a quoted query needed at least five posts to count as usable.

03The headline

Keyword typeResult
Multi-word keywords tested8
Usable quoted (≥5 exact-match posts)6 (75%)
Fell back to loose matching2
Single-word (quoting changes nothing)2

Six of eight multi-word keywords returned enough exact-phrase results to use directly. The two that underflowed fall back safely to loose matching, so no keyword loses coverage.

04The pattern: do humans type this phrase?

The keywords that underflowed shared a shape — three loosely-connected concepts nobody types verbatim, like “dating communication skills” or “online dating struggles”. People write “I’m struggling with online dating”, not the noun phrase.

The ones that worked were real idioms people post as-is: “communication in relationships”, “bad at flirting”. So the predictor of success isn’t word count — it’s whether the phrase is something a human actually types.

05What we shipped

Quoted-with-fallback: for any multi-word keyword, try the exact phrase first; if it clears the five-post bar, use it; otherwise fall back to loose matching. It costs about one extra search call on the keywords that need the fallback — roughly 1.25 calls per multi-word keyword on average.

Caveats we keep in view

Small sample (eight multi-word keywords, two domains), and Reddit’s search ranker shifts over time, so individual counts near the threshold can flip. The 75% verdict is directionally solid, not a precise constant.

How the pipeline works →

Frequently asked questions

Wrap the phrase in straight double quotes — “communication in relationships” — and Reddit returns posts that contain those words in that order, instead of treating them as a loose bag of words. It works well for phrases people actually type as-is. For three loosely-connected concepts nobody writes verbatim, the quoted query can underflow, so a sensible search falls back to loose matching.

Often, yes. In our A/B test about 75% of multi-word phrases returned enough exact-phrase matches to use directly — quoting cut the off-topic noise without starving the result count. The exception is awkward noun phrases like “online dating struggles” that humans rarely type word-for-word; those underflow and read better with loose matching as a fallback.

Loose, tokenized search treats your phrase as a bag of words and pulls in anything vaguely adjacent — search “dating communication skills” unquoted and the top hit can be a drama subreddit whose posts never contain the phrase. Quoting the phrase forces exact-order matching, which sharply raises precision when the phrase is something people genuinely post.

Keyword (tokenized) search maximises recall but drags in noise; quoted search maximises precision but can underflow on phrases nobody types verbatim. We use quoted-with-fallback: try the exact phrase first, keep it if it clears a small results bar, otherwise fall back to keyword matching. About 75% of multi-word phrases stay quoted, so no keyword loses coverage.

Keep reading

Related guides & use cases.

Validate what people actually say, not what you wish they would.