I was curious how much the reviewers got. A little digging reveals that they get $10 per review.
UserTesting.com - 15 minute video of a user testing your site for $19
11–20 of 30 posts
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#12Earlier quoted context omitted.
I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?
Why? The main deliverable in user testing is quality . It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.
A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment.
A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data.
Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#13I've used it - really helpful. Wish they would send their testers eye tracking monitors - that would rock.
Agreed, but do they not show the cursor?
Also, do they have a "more users, fewer minutes" option? I know I'd rather have 15 users for one minute each, or seven for two minutes each. After that, most users will have decided to stay or flee any ways. It would be good to get a sense of that early decision-making, especially across a statistically-reliable sample.
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#14Can anybody share a video session that they themselves commissioned? I'd like to see how the quality of feedback compares with the "official" demo of tripadvisor.
P.S. Somebody else mentioned eye tracking - I think commercial solutions for this are pretty expensive, but I've seen a few hobbyists roll their own using low cost webcams, though I'm not sure how they fair in terms of accuracy.
P.P.S. And somebody else mentioned just embedding a screen grabber directly onto your website - I'm sure it's achievable but with a browser extension, i.e. Mozilla. I'll have a play around and see what I can come up with.
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#15How do you guys think they find testers? It's specific to demographic and such, so i think it's beyond mechanical turk?
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#16We've used it in the past, and will in the future - BUT this is not rigorous usability testing of the sort that you'll get in the lab. They're self-selected and they don't do well with loosely defined objectives (even though they tend to be North American native English speakers).
If you have access to a Mac and can recruit your own volunteers (college campus, etc), I'd recommend getting Silverback - http://silverbackapp.com/ . Not only do you get the screen capture, you also get to see frustration/delight on users' faces. Plus, it's cheaper if you plan on doing four or more tests.
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#17Earlier quoted context omitted.
I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?
Why? The main deliverable in user testing is quality . It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.
I think there's room for both. Turkers might simulate the casual browsers who happen upon your site. There could be valuable data buried in what they do in a short period of time.
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#18I'll do this for $10. :p
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#19"Don't Make Me Think" ( http://www.squeezedbooks.com/book/show/9/dont-make-me-think-... ) has a good section on cheap usability testing.
Re: UserTesting.com - 15 minute video of a user testing your site for $19
#20Earlier quoted context omitted.
Why? The main deliverable in user testing is quality . It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.
A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment. A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data. Feedback from people whose jo…
Probably depends on what you're looking for. When you're just starting out a site, just a couple users will point out all sorts of user interface weaknesses that'll keep you busy fixing for a long while.