Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mechanical Turk shutting down September 30

mturk.com

Post reply on HN
Print view View original

141–150 of 177 posts

Re: Mechanical Turk shutting down September 30

#141

Post by internet_points »

loremm wrote: post #20
x0xMaximus wrote: post #13

Earlier quoted context omitted.

Initially the annotation of biomedical literature ( https://pubmed.ncbi.nlm.nih.gov/25592589/ ) but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT's tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space My favori…

I truly have no idea what this website is saying https://generalresearch.com/

> Our Customers

> Ripe with fraud, labor exploitation, political polling manipulation; our customers demand the best tools and security for reaching global audiences and securing respondent reliability at any scale.

wat

Re: Mechanical Turk shutting down September 30

#143

Post by vavos »

inigyou wrote: post #82
madrox wrote: post #39

Earlier quoted context omitted.

This is what I meant by full stack AI companies. I don't think you could get humans into the loop fast enough if they didn't have some idea of the type of task involved. I don't want people to be asked to fold a tshirt one moment and do a difficult traffic merge the next.

It doesn't really matter what you want though, only what CEOs want and that's low costs. I can see a combined shirt folding/traffic merging platform taking off.

but call center are still mostly single client even though it would be cheaper for any worker to be able to answer to any call. So clearly the expertise and context trade off is too big to be worthwhile

Re: Mechanical Turk shutting down September 30

#144

Post by yard2010 »

donalhunt wrote: post #126

Worth noting that one of the most high profile uses of Mechanical Turk was in September 2007 to review satellite imagery in a massive crowdsourced effort to find missing record-setting aviator Steve Fossett. The effort failed to find any areas of interest and the missing aviator was found the following year by a hiker. I wonder if there was any analysis after the fact to understand if the imagery actually provided an…

> Amazon's search effort was shut down the week of October 29, without any measurable success. Major Cynthia Ryan later said it had been more of a hindrance than a help. She said that persons purporting to have seen the aircraft on the Mechanical Turk or have special knowledge clogged her email during critical days of the search, and for even months afterward. Many of the ostensible sightings proved to be images of CAP aircraft flying search grids, or simply mistaken artifacts of old images. Psychics flooded the search base in Minden with predictions of where the aviator could be found. One man from Canada was particularly persistent with daily calls to Ryan. Ryan noted that every message, letter, or phone call was taken seriously, which swamped the USAF specialists assigned the task of reviewing every one of them without regard to apparent plausibility. In retrospect, the crowdsource effort was "not ready for prime time", according to Ryan

From Wikipedia

Re: Mechanical Turk shutting down September 30

#145

Post by nerevarthelame »

A lot of the early tasks posted to Mechanical Turk by Amazon to get people using their new platform were easily automated. You could basically choose the "none of the above" option 100% of the time and end up with a quality score above their threshold.

Instead of partying in college, my friends and I wrote a script to complete the tasks. We took over university computer labs to run it on a bunch of different computers. We made a few thousand bucks. Good times.

Re: Mechanical Turk shutting down September 30

#146

Post by ShinyLeftPad »

Nition wrote: post #78
x0xMaximus wrote: post #13

Earlier quoted context omitted.

Initially the annotation of biomedical literature ( https://pubmed.ncbi.nlm.nih.gov/25592589/ ) but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT's tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space My favori…

> Can you say what your use of it was? > Initially the annotation of biomedical literature but left academia for commercial use where we transitioned to arbitrage the market research and ephemeral task markets. It was essentially a research panel with a underdeveloped UI for tasks.. so abstract away AMT's tooling so it can be used by any buyer in the ResTech, Political Polling and Data Annotation space. Reading this…

> Reading this is similar to how I feel when I've asked Claude about something it coded for me

I didn't feel like that at all.

Re: Mechanical Turk shutting down September 30

#147

Post by ShinyLeftPad »

rkozik1989 wrote: post #138
owebmaster wrote: post #58

Earlier quoted context omitted.

20% of 0.07 was billed as 1 cent, not 1.4 nor rounded up to 2.

Imagine your favorite feature of a product was spend 1 cent on a real human being time and effort rather than 2 cents.

Or spend the same and let the human earn more?

Re: Mechanical Turk shutting down September 30

#148

Post by aftbit »

rkozik1989 wrote: post #138
owebmaster wrote: post #58

Earlier quoted context omitted.

20% of 0.07 was billed as 1 cent, not 1.4 nor rounded up to 2.

Imagine your favorite feature of a product was spend 1 cent on a real human being time and effort rather than 2 cents.

That's on the commission that Amazon took, not on the human.

Re: Mechanical Turk shutting down September 30

#149

Post by yellow_lead »

internet_points wrote: post #141
loremm wrote: post #20

Earlier quoted context omitted.

I truly have no idea what this website is saying https://generalresearch.com/

> Our Customers > Ripe with fraud, labor exploitation, political polling manipulation; our customers demand the best tools and security for reaching global audiences and securing respondent reliability at any scale. wat

Misplaced modifier? Wow

Re: Mechanical Turk shutting down September 30

#150

Post by ealready_value »

About 8 years ago, we used mturk for reading data out of public PDFs generated by a huge range of producers. I am not sure that LLMs would have been able to consistently extract this data until recently as a good number of these PDFs were scans, sometimes a scan of scan.

We had pretty good luck in the end, but getting there produced a somewhat large app on our side that would manage the whole process, including but not limited to asking for multiple responses, comparing them to each other, finding consensus, and determining which users would consistently produce bad responses and stop them from responding. We got to a confidence that about 85-95% of the data was correct, which was good enough for the company.

Through that process, I learned a couple things about managing mturk, primarily about how changes to the cost-per-task would change the process. Initially, we thought that price would be a quality knob, but quickly learned that price was a speed know. The higher the price the faster the tasks would be taken and completed. Quality did not change significantly as the price went up or down.

Overall, I still have fondness to mturk, but it was really bare-bones experience that needed a lot of work to get working effectively.

Post reply on HN