Usability Testing on a Budget: Five Studies, Real Findings

The maths behind small rounds is public, which is why it survives every budget meeting. Nielsen Norman Group's 2000 explanation of the five-user rule reports that each participant typically uncovers about 31 percent of the problems in a flow, which is why five people together reach roughly 85 percent of everything a round can expose.

That number is a floor, not a law. The group's 2012 review of sample sizes by method puts a quantitative study at 20 participants or more, eye-tracking at 39, and card sorting at 15 or more for every group; it also records rounds of as few as two users on the cheapest projects, and an average of 11 participants per round in a survey of 217 practitioners.

The habit worth breaking is buying credibility by size: one study, fifteen or twenty participants, a thick appendix. Three rounds of five beat it, because a study that large repeats its own first hour many times while a later round tests a design that has already been repaired.

Budget follows the same logic. Five users is a recruiting problem before it is a research problem: the work is finding people who did the task this month, watching them use the live build, and shipping the fix before the next round is booked. Sample size is the part everyone negotiates; participant behaviour is the part that decides whether the round was worth running at all.

Two free datasets catch problems that no session would ever be booked for. Automated scanning flags contrast, missing labels and structure on every deploy, and field data shows what real visits experienced rather than what a laptop in an office experienced. Neither one replaces a user, and both are cheaper than the meeting where someone argues that a problem is rare.

Response time is the oldest instrument of all. Nielsen Norman Group's 1993 limits still describe what a slow screen costs: a tenth of a second reads as instant, one second keeps a person's flow intact, and ten seconds is the edge of attention; past that point the interface owes the user a progress indicator rather than patience.

1. 01

   Recruit behaviour, not demographics

   Pick five people who completed the task during the past month, not five who match a persona. The round is only as good as the behaviour in the room, and a panel recruited for age and city will happily discuss a product they have never used.

   One week of recruiting — removes the risk of testing opinions about the product

2. 02

   Run the heuristic pass before booking anyone

   Three to five evaluators, passes of one or two hours, findings written as problems rather than missing features. Fix what needs no participant, then spend the round on the questions that only a participant can answer.

   One afternoon per evaluator — removes the items that need no user at all

3. 03

   Watch five users live, then ship a fix

   Book the round against the current build, attend every session, and have the fixes scheduled before the next round is booked. A round whose findings enter a backlog unchanged is a round the team has already decided to ignore.

   One week — removes the risk of a report nobody acts on

4. 04

   Put the free instruments on a monthly loop

   Automated scans and field vitals, reviewed in a fifteen-minute meeting with a named owner. A dashboard nobody opens is a subscription, not a process.

   Monthly — removes the risk of finding a broken form a quarter late

A testing programme is mostly sitting still and writing down what happened. The three photographs below are about the surfaces the method depends on: notes on paper, a device used by touch, and a calendar that decides whether anyone can attend at all.

1. Run the first round at five participants on the live build.Five people reach roughly 85 percent of what a round can find, at an average per-person detection rate near 31 percent (Nielsen Norman Group, 2000).
2. Book repetition rather than a bigger sample.Three rounds of five out-find a single study of fifteen, because each round tests a repaired design instead of repeating the first one.
3. Open with the method that needs no participants.A heuristic pass uses three to five evaluators and one or two hours per pass (Nielsen Norman Group, 2023).
4. Size the study to the method, not to habit.Quantitative work starts at 20 participants, card sorting at 15 per group and eye-tracking at 39 (Nielsen Norman Group, 2012).
5. Read the free scans every month.WebAIM's 2026 scan of a million home pages found detected WCAG 2 failures on 95.9 percent of them, at an average of 56.1 errors per page.
6. Publish the response-time limits next to the design.A tenth of a second reads as instant, one second keeps the flow, and ten seconds is where a progress indicator stops being optional (Nielsen Norman Group, 1993).