Method comparison

Reddit vs surveys vs user interviews

Three ways to learn what customers want, each strong where the others are weak. Here’s how they compare — and a simple rule for choosing.

Three ways to ask the same question

If you want to know what customers want, you have three workhorse methods: send a survey, run user interviews, or mine what people already say in public. They’re often treated as rivals, but they capture genuinely different things — and the smart move is matching the method to the question, not picking a favourite.

How they compare

DimensionSurveysUser interviewsForum mining
What it capturesStated preference, at scaleStated preference, in depthCloser to revealed behaviour
RecruitingNeed a list or panelSchedule each personNone — already posted
ScaleHundreds, easilyFive to fifteen, realisticallyHundreds of threads
Main biasLeading questions, agreeablenessSmall n, interviewer effectSkews frustrated & engaged
DepthShallow, fixed answersDeepest — you can probeMedium — rich but no follow-up
Cost & speedCheap if you have a listSlow, labour-heavyCheap and fast

No method is “best” — each is strong exactly where another is weak. Forum mining can’t ask a follow-up; an interview can’t reach hundreds; a survey can’t catch what people didn’t think to mention.

When each one wins

Reach for a survey when you already have an audience and a sharp, closed question — “which of these three do you use?” — and you want it quantified across many people.

Reach for interviews when the why matters more than the how-many: untangling a confusing workflow, understanding a decision, exploring something you can’t yet phrase as a question. Depth is their whole point.

Reach for forum mining first when you don’t have an audience yet, when you need breadth fast and cheap, or when you specifically want what people say unprompted rather than what they tell a researcher.

They’re better together

The strongest research stacks them. Mine forums to find the real language and the candidate problems, use that to write a survey that asks the right closed questions, then interview a handful of people to understand the why behind the numbers.

Forum mining is usually the cheapest place to start because it needs no recruiting and surfaces the questions worth asking — so the surveys and interviews that follow aren’t built on your guesses about what matters.

Why unprompted talk is different data

The deeper reason forum mining complements the others: it’s closer to revealed preference than to stated.

Revealed vs stated preference

Validate what people actually say, not what you wish they would.