Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You Only Need to Test with 5 Users (2000)

nngroup.com

Post reply on HN
Print view View original

81–90 of 112 posts

Re: You Only Need to Test with 5 Users (2000)

#81

Post by hycaria »

wwweston wrote: post #59
derp_dee_derp wrote: post #57

Earlier quoted context omitted.

Ux Engineering manager here for a fortune 100 company. We don't care about any of the use cases in your comment. If our JavaScript doesn't work in your browser, you are a security risk and we don't want our site to work on your browser. Please call our 1-800 number to talk to a representative.

> We don't care about any of the use cases in your comment Hey, everybody has to make tradeoffs, and it's certainly conceivable that the margins of disability don't matter to you, or that your phone support is your non-visual UA, or even that you don't care about search. That's only half the contents of my comment, though, the rest is about engineering benefits. Perhaps you don't care about those either? > If our Jav…

> it's just clear that there's opportunities to rise well above one's competence there.

Isn't that super hostile ? Can't read between the lines there

Re: You Only Need to Test with 5 Users (2000)

#82

Post by eitland »

collyw wrote: post #78
eitland wrote: post #71

Earlier quoted context omitted.

I can only speak for myself, but for me who grew up with C64, Windows 3.1, 95, 98, Linux, Windows XP more Linux etc that icon still didn't make sense until I got it explained. Then again, the magnifykng glass icon for search didn't make sense either until I read the docs but at least back then people made docs and kept the UI stable enough so that it made sense to learn it. I still remember fondly being good with OLE…

The old days seemed so much more intuitive when there was a crappy low res icon with text underneath.

Not sure if you are sarcastic and in what way, but if I understand you correctly you think we are better off now.

I think we would be better off if we kept the best from both: the predictability and discoverability of the past with the niceness of today.

Why should we have to choose between nice and usable?

Re: You Only Need to Test with 5 Users (2000)

#83

Post by tobbebex »

Etheryte wrote: post #74
bravura wrote: post #49

Earlier quoted context omitted.

You don't have $10? Come on.

This might sound condescending, but it is a good point. Anyone can be a test user, you don't need to formally hire people with a contract etc (unless your work is contractually classified, but that's a whole other bag of beans). You can ask colleagues from other departments, people at a coffee shop, friends etc.

It depends on the userbase. I write medical imaging software intended to be used by MDs, and can’t just ask a random person at a coffeshop to try to establish a certain cancer diagnosis using our software. Finding candidates that have time for usability testing in that target group can be quite a challenge.

Re: You Only Need to Test with 5 Users (2000)

#84

Post by dalbasal »

The caveat here is that it depends on what you mean by "test" and what "insights" you are interested in.

If you are interested in (for example) determining the optimal price and only 1/20 users buy something... you still probably need about 1,000 X (price points you want to test).

In that "test" you are basically trying to uncover the demand curve (or points on it). It's a statistical question.

Say you have a dating app, and you are trying a new matching algorithim. It will also take thousands of matches before you have the data to make determinations.

All that said, I totally agree with the author. I would just frame it differently.

The question you need to ask is "do I need statistics?" Statistics have become habitual, but much (most?) of the time, we don't need statistics.

If you want to learn if a user can write and publish a blog using your software or install a water filter under they sink... you don't need statistics. You need to know where most people get into trouble, and n=5 will work fine for that.

This is intuitive if we just think outside of the "testing" vocabular. You write a CV/essay/article. You ask 1-3 friends to read it and advise. You don't produce statistics.

Re: You Only Need to Test with 5 Users (2000)

#85

Post by blue_devil »

Usability testing is more like brainstorming. Those 3-5 users may tell you what some usability issues are - but they won't tell you how big those issues are, for your purposes. With imperfect a priori information on who the users of your thing are, you have to go back to assumptions.

Re: You Only Need to Test with 5 Users (2000)

#86

Post by julienreszka »

dalbasal wrote: post #84

The caveat here is that it depends on what you mean by "test" and what "insights" you are interested in. If you are interested in (for example) determining the optimal price and only 1/20 users buy something... you still probably need about 1,000 X (price points you want to test). In that "test" you are basically trying to uncover the demand curve (or points on it). It's a statistical question. Say you have a dating…

you probably don't but you probably should

Re: You Only Need to Test with 5 Users (2000)

#87

Post by buro9 »

I always worked on the premise: "Find five people who care"

It's nice to get the "five" verified, but I still think it's important to make sure they care, to have them be an essential part of making the product better. The trick though is not to be drawn into making anything for any one of these users (it's still got to be a generic usability for all users).

Re: You Only Need to Test with 5 Users (2000)

#88

Post by TrialError »

I have loved this rule ever since I came across it as a student in 2002 and been using it successfully for user acceptance testing strategies in large projects.

I interpret it as a sort of fixed Pareto principle for projects where you limit effort and team size for maximum gain. N=5 also happens to be close to the ideal team size in agile frameworks. This rule is smart in a lot of ways and was ahead of its time.

Re: You Only Need to Test with 5 Users (2000)

#89

Post by adrianN »

saagarjha wrote: post #80
derp_dee_derp wrote: post #57

Earlier quoted context omitted.

Ux Engineering manager here for a fortune 100 company. We don't care about any of the use cases in your comment. If our JavaScript doesn't work in your browser, you are a security risk and we don't want our site to work on your browser. Please call our 1-800 number to talk to a representative.

> If our JavaScript doesn't work in your browser, you are a security risk and we don't want our site to work on your browser. Sorry, what? Why am I security risk for not wanting to run the arbitrary code that your website sends me? (I browse with JavaScript turned on, FWIW, so this is a hypothetical question for me.)

Some javascript is used to detect bots.

Re: You Only Need to Test with 5 Users (2000)

#90

Post by pvinis »

mclehman wrote: post #47
HeWhoLurksLate wrote: post #33

Earlier quoted context omitted.

One of my friends was literally hired as an intern to try to break software last year. He loved it, and found a ton of bugs, which was really helpful to the company- they eventually paid him a $1,000 bonus for his help over the summer.

I have a friend who does this for free. Every single thing I've ever shown him (both my software and that of others) breaks as soon as he gets his hands on it. He's the sort to dig as far into the internals of things as he can and then start messing with it (he implemented partial function application for C, for example [1]). My poor prototypes never stood a chance. [1] https://github.com/zwimer/C-bind

That's so enjoyable to do. :D
Post reply on HN