Every usability practitioner, early or later, run into a simple question: how many users I need to conduct a good usability test? Jacob Nielsen argue in his alertbox that five users are enough to detect 80% of all usability problems. On the other hand Christine Perfetti and Lori Landesman claims that eight people are not sufficient to discover all usability problems. So what is the answer to our question?
Before giving an answer to our question we have to take in considerations some factors that influence the number of people that had to be hired in order to conduct usability tests:
The biggest obstacle for usability is represented by project budget. Here, in Italy, companies does not invest on usability and all related subjects are treated as optional services in estimates. Clients infact prefer get discounts in the final price by cutting off usability costs (including user testing) or more frequent investing available budget in other services like home page flash animations. These prejudices toward usability influence in the negative the final product and when clients realize of the mistake they made, is too late to find solutions without investing more money. So there's no budget to hire people and conduct usability tests.
Clients believe that good web site can be done in a week and insists to get it published in five days: one day to get material, one day to prepare layout, two days to develop pages and last day to make funcionality test and deploy site. There's no option to consider usability studies in such timeline. So there's no time to hire people and conduct usability tests.
To analyze this factor I will take as an example web sites projects that can be divided into four types (naturally this is a simplified categorization):
Logically usability analysis is useless in small size web site due to their small complexity and most important for their reduced budget. In medium size web sites and in web applications usability analysis can be done either using an heuristic evaluation or usability test with no more than five users. Naturally, if funds and time are sufficient is better use both options. In large web sites depending on budget (timelines in this kind of web sites in generally more relaxed) is better conduct heuristic evaluation and usability test with no more than ten users.
So depending on the external factors the best result can be achieved making usability test with no more than ten people in most cases closer to five. Moreover is better conduct usability tests integrating them with an iterative approach ideally following this procedure: