Live data from Hacker News

Tell HN: Mechanical Turk is twenty years old today

news.ycombinator.com

1–10 of 65 posts

Tell HN: Mechanical Turk is twenty years old today

#1
MTurk was built by two two-pizza teams at AWS over the course of a year and launched on Nov 2, 2005. It took a few days for people to find it and catch on, but then things got busy.

At the time, AWS was about 100 people (when you were on call, you were on call for all of AWS), Amazon had just hit 10,000, S3 was still in private beta, and EC2 was a whitepaper.

What did you create with MTurk and the incredibly patient hard-working workforce behind it?

Re: Tell HN: Mechanical Turk is twenty years old today

#4
My wife had dozens, well probably over 100, handwritten recipes from a dead relative. They were pretty difficult to read. I scanned them and used mturk to have them transcribed.

Most of the work was done by one person - i think she was a woman in the Midwest, it's been like 15-years so the details are hazy. A few recipes were transcribed by people overseas but they didn't stick at it. I had to reject only one transcription.

I used mturk in some work projects too but those were boring and maybe also a little unethical (basically paying people 0.50 to give us all of their Facebook graph data, for example.)

Re: Tell HN: Mechanical Turk is twenty years old today

#5
We asked user to evaluate 300x300 pixel maps. Users were shown two image and had to decide which better matched the title we chose. Answers were something like "left", "right", "both same", "I don't know". Due to misconfiguration the images didn't load for users (they only loaded in our internal network). Still we got plenty of "left" and "right" answers. Random and unusable. Our own fault of course.

Re: Tell HN: Mechanical Turk is twenty years old today

#6
I assisted many professors in data collection for their research in the grad school. Later I also collected data for a couple of my papers. mTurk was very popular in the beginning in large part due to its low cost. Then one day they jacked up their commission so much, it was no more attractive to me. Also, the response/task quality went down significantly. My last time using it was in 2018 for a large scale image labeling task. After doing a pilot run, I concluded I was getting garbage. I went to some other vendor and never returned to mTurk after that.

Re: Tell HN: Mechanical Turk is twenty years old today

#7
I supported data curation on it in the beginning but it became a popular way to exploit labor. I really love the idea but the main value is specifically to gain advantage over wealth inequality. I really support MTurk and the hard workers on it but I also cannot ignore the negatives.

Re: Tell HN: Mechanical Turk is twenty years old today

#8
Never used it since it was only available in the US. Looks like additional countries were not added till Oct 2016, 11 years after it was first launched[0]

None of the companies I've worked for have used it AFAIK, despite them all using AWS. I think I've mostly ignored it as one of the niche AWS products that isn't relevant.

[0] https://blog.mturk.com/weve-made-it-easier-for-more-requeste...

Re: Tell HN: Mechanical Turk is twenty years old today

#10
Using the Propser.com data set (a peer-to-peer lending market), I used MTurk to analyze the images of people applying for a loan. This was used in a finance research project with 3 University of Washington professors of Finance.

The idea was that the Prosper data set contained all of the information that a lending officer would have, but they also had user-submitted pictures. We wanted to see if there was value in the information conveyed in the pictures. For example, if they had a puppy or a child in the picture, did this increase the probability that the loan would get funded? That sort of thing. It was a very fun project!

Paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1343275

Post reply on HN