What is quantitative user research?

What counts as quantitative research

Anything that produces numbers from a reasonably large group of users qualifies. The common sources are:

• Analytics: page views, completion rates, drop-off at each step, time on task.
• Surveys with scaled or closed questions, such as satisfaction ratings.
• A/B tests comparing two versions of a design.
• Unmoderated usability tests that record success rates and task times.
• Card sorting and tree testing at larger sample sizes.

The common thread is that you can count, compare and, with enough data, test whether differences are real or just luck.

What it is good at

Numbers are excellent at showing scale and change. They tell you that 40 percent of shoppers abandon at the shipping step, or that a redesign cut the time to complete sign-up. They let you compare segments, spot trends across months and check whether a change really moved a metric.

They also carry weight in meetings. A single vivid quote can be dismissed as an anecdote, but a pattern across thousands of sessions is harder to wave away.

What it cannot tell you

A number says what happened, not why. If completion drops after a redesign, the data points at the problem and says nothing about the cause. Maybe the button moved, maybe the copy confused people, maybe a competitor launched a sale.

That is the reason most mature teams use both approaches. Quantitative methods find and size the problem, qualitative ones explain it. Using only one is like reading a map without ever walking the streets, or walking the streets without ever opening the map.

How to run it properly

Start with a clear question and decide in advance which metric will answer it. Define who you are measuring, since an average across very different users can hide both groups' reality.

Then get a sample large enough to trust. How large depends on the method and the size of the difference you are hoping to detect, so use a sample size calculator rather than guessing. Randomize where you can, avoid leading survey questions and decide on your analysis plan before you see the data, not after.

Finally, report ranges and caveats honestly. A result with wide uncertainty is still useful if you say so.

Pitfalls to avoid

Tracking everything and learning nothing is common. So is chasing a metric that is easy to measure rather than one that matters, like clicks instead of successful outcomes. Be wary of tiny samples presented as statistics, and of small differences that are not statistically meaningful. And remember that a metric you reward will be gamed, even unintentionally.