Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The $5 Guerrilla User Test

blog.bumblebeelabs.com

Post reply on HN
Print view View original

31–40 of 49 posts

Re: The $5 Guerrilla User Test

#31

Post by shalmanese »

jdrock wrote: post #19
Spikefu wrote: post #15

Earlier quoted context omitted.

FTA: "Bars are wonderful at segmenting by demographic. Match the bar you’re going to with the user population you want to target. Different bars will produce slightly different results but the variation is not huge."

Yeah, I don't know a bar frequented by people interested in the Semantic Web or web-crawling.

Sophisticated users will reveal sophisticated usability flaws in your application but even unsophisticated users can reveal the unsophisticated flaws.

One really great tip (for sober testing, not drunk) is to use role playing. If you were soliciting feedback for a blog analytics package for example, you could tell users "Imagine you're the CEO of a lollipop manufacturer". Most people end up getting really into the role you're asking them to play and ask surprisingly insightful questions.

That being said, you still need to test with sophisticated users but they're much harder to get and their time is expensive so it's always a good idea to test with a few unsophisticated users first and fix the obvious usability flaws so you can maximize the amount of relevant feedback you do get from your sophisticated users.

Re: The $5 Guerrilla User Test

#32

Post by ThomPete »

forcer wrote: post #4
ThomPete wrote: post #3

I call BS on this. It has no usage what so ever. You can't test for success you can only monitor it.

Why no usage what so ever? Any process of getting user's feedback is better than doing nothing. We are so skewed of using our own apps that we think all is perfect. Nothing ever is. I think its a great idea how to get more usage data. If I try it I will let you know how it went. I am thinking of using some screenrecording software like SilverBackApp to archive and analyze results later when not drunk :) Janusz Founde…

Because success is not measured by the usability of your product.

There are countless examples of way too complex products that are still successful and almost any successful company you can think of DIDN'T do usability testing.

Companies like Amazon do A/B testing, Apple don't do usability testing at all.

Usability starts to matter later on, when you can measure on actual customer behavior rather than user opinion.

Down vote me all you want. I challenge anyone to prove that usability tests are actually making more successful products.

Re: The $5 Guerrilla User Test

#33

Post by ThomPete »

ynniv wrote: post #23
ThomPete wrote: post #3

I call BS on this. It has no usage what so ever. You can't test for success you can only monitor it.

I notice that you are a design professional and have written lengthy articles on interface design. The point of this article is that buying a handful of beers will improve your product design immensely, when compared to doing nothing at all. It is an amateur move that probably looks useless from a professional standpoint, but like 24-grit sandpaper, makes a rough job look more polished.

Do you really think it's that simple?

It's not.

There is a whole list of issues with asking people about what they think.

Again success of a product is not secured by these kind of tests. Sure if you are doing something that is outright useless but then asking anyone (like girlfriend) is enough.

What he is suggesting is equivalent of doing code review with a first year programmer.

Wouldn't you call BS if I suggested something like that?

I am telling you as a professional, that's not any way to go about testing your product. Unless it's outright useless. But then you don't have to spend 5USD on a beer. You can ask ANYONE for free.

Opinions are not valuable feedback.

Re: The $5 Guerrilla User Test

#34

Post by shalmanese »

ThomPete wrote: post #32
forcer wrote: post #4

Earlier quoted context omitted.

Why no usage what so ever? Any process of getting user's feedback is better than doing nothing. We are so skewed of using our own apps that we think all is perfect. Nothing ever is. I think its a great idea how to get more usage data. If I try it I will let you know how it went. I am thinking of using some screenrecording software like SilverBackApp to archive and analyze results later when not drunk :) Janusz Founde…

Because success is not measured by the usability of your product. There are countless examples of way too complex products that are still successful and almost any successful company you can think of DIDN'T do usability testing. Companies like Amazon do A/B testing, Apple don't do usability testing at all. Usability starts to matter later on, when you can measure on actual customer behavior rather than user opinion.…

Both Amazon & Apple do usability testing. Apple doesn't do focus groups which are an entirely different thing.

Re: The $5 Guerrilla User Test

#35

Post by shalmanese »

ThomPete wrote: post #33
ynniv wrote: post #23

Earlier quoted context omitted.

I notice that you are a design professional and have written lengthy articles on interface design. The point of this article is that buying a handful of beers will improve your product design immensely, when compared to doing nothing at all. It is an amateur move that probably looks useless from a professional standpoint, but like 24-grit sandpaper, makes a rough job look more polished.

Do you really think it's that simple? It's not. There is a whole list of issues with asking people about what they think. Again success of a product is not secured by these kind of tests. Sure if you are doing something that is outright useless but then asking anyone (like girlfriend) is enough. What he is suggesting is equivalent of doing code review with a first year programmer. Wouldn't you call BS if I suggested…

You seem to have a mistaken idea of what user testing is. It's not asking people what they think of this software, it's asking them to perform specific tasks and then observing where they make mistakes or have issues.

The benefits of user testing are well documented. You're most definitely in the minority position for claiming this.

Re: The $5 Guerrilla User Test

#36

Post by ThomPete »

ThomPete wrote: post #32
forcer wrote: post #4

Earlier quoted context omitted.

Why no usage what so ever? Any process of getting user's feedback is better than doing nothing. We are so skewed of using our own apps that we think all is perfect. Nothing ever is. I think its a great idea how to get more usage data. If I try it I will let you know how it went. I am thinking of using some screenrecording software like SilverBackApp to archive and analyze results later when not drunk :) Janusz Founde…

Because success is not measured by the usability of your product. There are countless examples of way too complex products that are still successful and almost any successful company you can think of DIDN'T do usability testing. Companies like Amazon do A/B testing, Apple don't do usability testing at all. Usability starts to matter later on, when you can measure on actual customer behavior rather than user opinion.…

shalmanese

Amazon do A/B tests.

Apple doing usability tests? Not according to those I know there. Don't you mean testing?

Re: The $5 Guerrilla User Test

#37

Post by ThomPete »

shalmanese wrote: post #35
ThomPete wrote: post #33

Earlier quoted context omitted.

Do you really think it's that simple? It's not. There is a whole list of issues with asking people about what they think. Again success of a product is not secured by these kind of tests. Sure if you are doing something that is outright useless but then asking anyone (like girlfriend) is enough. What he is suggesting is equivalent of doing code review with a first year programmer. Wouldn't you call BS if I suggested…

You seem to have a mistaken idea of what user testing is. It's not asking people what they think of this software, it's asking them to perform specific tasks and then observing where they make mistakes or have issues. The benefits of user testing are well documented. You're most definitely in the minority position for claiming this.

I am quite aware of what a usability test is as I have spend quite some time observing them.

But as I have also learned over the years is that usability test does not equal more successful product.

You are welcome to prove the benefits claim.

I don't care if you can prove some academic self referencing point about usability testing improving customer sentiment. I care about successful products and there is absolutely nothing that proves that usability tests or the UCD method is better than any other design process.

What company's success is based on them doing usability tests. By all means, please share.

Re: The $5 Guerrilla User Test

#38

Post by shalmanese »

ThomPete wrote: post #36
ThomPete wrote: post #32

Earlier quoted context omitted.

Because success is not measured by the usability of your product. There are countless examples of way too complex products that are still successful and almost any successful company you can think of DIDN'T do usability testing. Companies like Amazon do A/B testing, Apple don't do usability testing at all. Usability starts to matter later on, when you can measure on actual customer behavior rather than user opinion.…

shalmanese Amazon do A/B tests. Apple doing usability tests? Not according to those I know there. Don't you mean testing?

Amazon do both A/B testing & usability testing.

Apple's own HIG advocates for usability testing: http://developer.apple.com/Mac/library/documentation/UserExp...

Do you have a cite that either does not do usability testing?

Re: The $5 Guerrilla User Test

#39

Post by shalmanese »

ThomPete wrote: post #37
shalmanese wrote: post #35

Earlier quoted context omitted.

You seem to have a mistaken idea of what user testing is. It's not asking people what they think of this software, it's asking them to perform specific tasks and then observing where they make mistakes or have issues. The benefits of user testing are well documented. You're most definitely in the minority position for claiming this.

I am quite aware of what a usability test is as I have spend quite some time observing them. But as I have also learned over the years is that usability test does not equal more successful product. You are welcome to prove the benefits claim. I don't care if you can prove some academic self referencing point about usability testing improving customer sentiment. I care about successful products and there is absolutely…

http://www.uie.com/articles/three_hund_million_button

Re: The $5 Guerrilla User Test

#40

Post by ThomPete »

shalmanese wrote: post #39
ThomPete wrote: post #37

Earlier quoted context omitted.

I am quite aware of what a usability test is as I have spend quite some time observing them. But as I have also learned over the years is that usability test does not equal more successful product. You are welcome to prove the benefits claim. I don't care if you can prove some academic self referencing point about usability testing improving customer sentiment. I care about successful products and there is absolutely…

http://www.uie.com/articles/three_hund_million_button

Ehh that is proving my point.

That's refinement AFTER launch of the product. That is something entirely different than what that 5USD test is all about.

Post reply on HN