Live data from Hacker News

Mechanical Turk shutting down September 30

mturk.com

81–90 of 178 posts

Re: Mechanical Turk shutting down September 30

#81
I was just thinking about the good old days combining human judgements on mturk for classifying high-value forum threads. Since replaced by using judgements from multiple LLMs.

The only use I can think of it these days is for social scientists who need to gather judgements from bona fide humans.

This will be devastating for a lot of workers in low-infrastructure countries.

Re: Mechanical Turk shutting down September 30

#82
post #39

Earlier quoted context omitted.

I think yes but on different categories. First one I imagine is robotics control and support. "This robot is having trouble folding a tshirt help it out for 1$" Unless they go the waymo route of highly trusted people but I think mass deployed robots are a bit safer than a car for this.

This is what I meant by full stack AI companies. I don't think you could get humans into the loop fast enough if they didn't have some idea of the type of task involved. I don't want people to be asked to fold a tshirt one moment and do a difficult traffic merge the next.

It doesn't really matter what you want though, only what CEOs want and that's low costs. I can see a combined shirt folding/traffic merging platform taking off.

Re: Mechanical Turk shutting down September 30

#83
post #20

Earlier quoted context omitted.

I truly have no idea what this website is saying https://generalresearch.com/

It kind of reminds me of those fake tech ads those guys put up in the NYC subway: https://www.instagram.com/p/DZacltqHMjT/?img_index=8&igsi=bT...

I can't see this because I don't have an Instagram account

Re: Mechanical Turk shutting down September 30

#84
post #20

Earlier quoted context omitted.

I truly have no idea what this website is saying https://generalresearch.com/

As usual, the real tldr is in the privacy policy https://generalresearch.com/supplier/privacy-policy/notice-t... > General Research Laboratories, LLC (“GRL”) is an aggregator of market research surveys in multiple marketplaces for business customers. GRL does not typically host consumer surveys which are conducted by other consumer-facing organizations.... So, just a middleman to give you fake/shady at best survey re…

Finally you get it. Only exception is that we run our own exchange now to do task bidding so we don’t need to deal exclusively with other companies to middleman. Core business is what’s called yield management (akin to DSP in adtech) where the best survey (is the user qualified for it, does it have the best pay, etc) is selected for traffic in <100ms. I’d only argue the shady companies are the ones paying proxies (like cint.com) and pushing paid user acquisition instead of surveys. We actively fund ontology development for better profiling targeting. but yes, big enterprises/consultancies can use and interpret the collected data however they want, not our responsibility and we have no legal rights over it anyway

Re: Mechanical Turk shutting down September 30

#85

Earlier quoted context omitted.

It makes me question why the surveys don't ask for the users country. That way they cd can figure it out without trying to guess it based off of IP.

Users/respondents lie, and many buyers/researchers are neo-luddites; most C level still come from the telemarketing days. A fun example: mobile targeting is terrible when off wifi because all survey platforms uniq identify users based off their IPv4, so any T-Mobile LTE users in the same city going through the same CGNAT often share profiling data that ends up conflicting, which ends up meaning they don't get sent in…

The fact that you can sell a product that doesn't work and become a billionaire proves capitalism is broken.

Re: Mechanical Turk shutting down September 30

#87
post #83

Earlier quoted context omitted.

It kind of reminds me of those fake tech ads those guys put up in the NYC subway: https://www.instagram.com/p/DZacltqHMjT/?img_index=8&igsi=bT...

I can't see this because I don't have an Instagram account

replace instagram.com with kittygr.am

Re: Mechanical Turk shutting down September 30

#88

It’s kind of crazy they’re shutting this down just when this Service probably has the most possibilities ever. You have an agent with Multiple people doing actual physical tasks in the real world seems like something that could be really powerful.

In spite of the name I think the overwhelming majority of the tasks were things that could be done by LLMs and they're probably getting flooded by people using bots to do exactly that. Even before the age of LLMs Mechanical Turk data was pretty bad because you'd have a bunch of people racing to answer questions as quickly as possible to get their $0.25 or whatever. So it was essentially a test of 'can you input random answers as quickly as possible while paying enough attention to notice the attention question that says to mark d.'

Any remotely open platform for this sort of stuff is going to wrecked by LLMs. But making some sort of high trust, high verification market is going to end up sending prices high enough that it becomes an unattractive proposition for use. And even in that case it's just going to be a cat and mouse game of people figuring out how to game the system enough to get trusted before handing it off to the LLM.

Re: Mechanical Turk shutting down September 30

#90
post #11

Mechanical Turk had a good run, but not surprised it's shutting down. I'm sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway. I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI…

I think yes but on different categories. First one I imagine is robotics control and support. "This robot is having trouble folding a tshirt help it out for 1$" Unless they go the waymo route of highly trusted people but I think mass deployed robots are a bit safer than a car for this.

There are already a number of companies providing RLHF and SFT services for AI providers that does a lot of validation/prequalification of people that'd be well placed to take on tasks like that, but the big problem to solve would be latency if you don't have people contracted to carry out a task right now.
Post reply on HN