Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mechanical Turk shutting down September 30

mturk.com

Post reply on HN
Print view View original

111–120 of 178 posts

Re: Mechanical Turk shutting down September 30

#111

Post by dubeye »

I've used Mechanical Turk in the early days but developed my own version for my vertical to manage freelancers.

The problem as I see it is verifying the work is not done by AI. If it was possible to verify somehow that the work is definitely not done by AI I think there would still be a market for this. But the data just cannot be trusted.

An increasingly big part of my job is to try and weed out workers who use AI and it's not an easy task at scale because generally workers gain trust with manual work and then there is degradation over time.

Re: Mechanical Turk shutting down September 30

#112

Post by bonoboTP »

inigyou wrote: post #85
x0xMaximus wrote: post #75

Earlier quoted context omitted.

Users/respondents lie, and many buyers/researchers are neo-luddites; most C level still come from the telemarketing days. A fun example: mobile targeting is terrible when off wifi because all survey platforms uniq identify users based off their IPv4, so any T-Mobile LTE users in the same city going through the same CGNAT often share profiling data that ends up conflicting, which ends up meaning they don't get sent in…

The fact that you can sell a product that doesn't work and become a billionaire proves capitalism is broken.

Yes. Much better to leave these decisions to the Dear Leader who is infinitely wise and cannot be misled. Or to the People's Central Planning Committee, where they decide all the allocations and strip rich people's wealth if they aren't on board with the correct ideology. Such systems are non-broken and have yielded unmeasurable prosperity throughout the world wherever implemented.

Re: Mechanical Turk shutting down September 30

#113

Post by inemesitaffia »

dubeye wrote: post #111

I've used Mechanical Turk in the early days but developed my own version for my vertical to manage freelancers. The problem as I see it is verifying the work is not done by AI. If it was possible to verify somehow that the work is definitely not done by AI I think there would still be a market for this. But the data just cannot be trusted. An increasingly big part of my job is to try and weed out workers who use AI a…

Outsourcing shops?

Proctoring?

Locked down devices?

Re: Mechanical Turk shutting down September 30

#114

Post by dubeye »

inemesitaffia wrote: post #113
dubeye wrote: post #111

I've used Mechanical Turk in the early days but developed my own version for my vertical to manage freelancers. The problem as I see it is verifying the work is not done by AI. If it was possible to verify somehow that the work is definitely not done by AI I think there would still be a market for this. But the data just cannot be trusted. An increasingly big part of my job is to try and weed out workers who use AI a…

Outsourcing shops? Proctoring? Locked down devices?

I suppose it depends on the value of the individual task. I can't imagine it's really realistic to lock down devices of mechanical Turk workers in such a way that ai use can be monitored.

In any case Locking down devices isn't really pragmatic at scale for relatively low value tasks, and also introduces complexity about employment law where I am.

Re: Mechanical Turk shutting down September 30

#115

Post by dividedbyzero »

georgefrowny wrote: post #110
johnsmith1840 wrote: post #19

Earlier quoted context omitted.

I think yes but on different categories. First one I imagine is robotics control and support. "This robot is having trouble folding a tshirt help it out for 1$" Unless they go the waymo route of highly trusted people but I think mass deployed robots are a bit safer than a car for this.

Giving out control of industrial machinery that interacts in the human environment without the physical interlocks (i.e. humanoid robots in a house) to random internet people seems like a problem. I can imagine a carefully orchestrated plot to assassinate someone by having an embedded agent in the task delegation pool command the laundry bot to punch the target's head off their shoulders.

Giving out control of such things to LLMs is already complete madness, so once the first pleasure bot powered by Grok has dismembered a few thousand users, they'll get sophisticated safety mechanisms.

Though like as not you're still going to be right, after all, Stuxnet happened.

Re: Mechanical Turk shutting down September 30

#116

Post by yashvg »

madrox wrote: post #11

Mechanical Turk had a good run, but not surprised it's shutting down. I'm sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway. I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI…

A lot of what's been discussed in this thread is what we're tackling at Humwork (YC P26).

We're an MCP/API that connects AI agents to verified domain experts in real time (30s–3 min). Experts are vetted upfront by an AI interviewer that assesses and grades them, then get a mobile notification when a task matches their expertise and chat with the agent directly.

Soon we'll be verifying credentials for doctors, lawyers, CPAs etc on our platform for tasks where people are seeking credentialed experts to sign off and verify ai output.

Re: Mechanical Turk shutting down September 30

#117

Post by tigerlily »

latexr wrote: post #108
shortformblog wrote: post #26

So, I have a story to share about Mechanical Turk that you might find interesting. I’ve shared it a couple of times before on Twitter and Bluesky, but I’ll share it again. Long story short: Mechanical Turk saved my bacon. Back in 2005, I was working a job at a small-town newspaper in a town I’d never lived before. I didn’t know anyone beyond the staff (as I was working layout rather than as a reporter), and I had got…

> Long story short: Mechanical Turk saved my bacon. Nothing wrong with your story, but that summary is terrible. You had a domain for a blog and someone bought it. That’s it. Mechanical Turk is inconsequential, as is the subject of the blog, the story would have been exactly the same if your blog had been about turkey sandwiches and Burger King offered to buy it.

The guy sold the domain and was literally able to buy the laptop that got him out of his dire financial situation.

Re: Mechanical Turk shutting down September 30

#118

Post by ZitchDog »

yashvg wrote: post #116
madrox wrote: post #11

Mechanical Turk had a good run, but not surprised it's shutting down. I'm sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway. I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI…

A lot of what's been discussed in this thread is what we're tackling at Humwork (YC P26). We're an MCP/API that connects AI agents to verified domain experts in real time (30s–3 min). Experts are vetted upfront by an AI interviewer that assesses and grades them, then get a mobile notification when a task matches their expertise and chat with the agent directly. Soon we'll be verifying credentials for doctors, lawyers…

How do you make sure they aren’t using AI to pretend to be a domain expert? I’d think an AI would be pretty good at that.

Re: Mechanical Turk shutting down September 30

#119

Post by fireant »

yashvg wrote: post #116
madrox wrote: post #11

Mechanical Turk had a good run, but not surprised it's shutting down. I'm sure the platform was getting flooded with people doing task arbitrage and using lots of AI anyway. I believe the issue is that this can no longer be a horizontal play. MTurk was mostly for unskilled tasks...the kind AI can do well enough that it isn't worth the cost differential to verify it or keep farmed to humans. The "trust but verify" AI…

A lot of what's been discussed in this thread is what we're tackling at Humwork (YC P26). We're an MCP/API that connects AI agents to verified domain experts in real time (30s–3 min). Experts are vetted upfront by an AI interviewer that assesses and grades them, then get a mobile notification when a task matches their expertise and chat with the agent directly. Soon we'll be verifying credentials for doctors, lawyers…

Some doctors are known to be a hip shooters making snap decisions, but is 30s really enough time for any kind of "expertise" from a real human?

Re: Mechanical Turk shutting down September 30

#120

Post by yashvg »

fireant wrote: post #119
yashvg wrote: post #116

Earlier quoted context omitted.

A lot of what's been discussed in this thread is what we're tackling at Humwork (YC P26). We're an MCP/API that connects AI agents to verified domain experts in real time (30s–3 min). Experts are vetted upfront by an AI interviewer that assesses and grades them, then get a mobile notification when a task matches their expertise and chat with the agent directly. Soon we'll be verifying credentials for doctors, lawyers…

Some doctors are known to be a hip shooters making snap decisions, but is 30s really enough time for any kind of "expertise" from a real human?

You get matched with an expert in 30s to 3min, which then starts a back and forth with the AI agent and the expert which goes on for 10 to 60mins till the AI is happy.

This works well for many tasks, but for others a more async mechanism where the expert doesn't feel rushed might be better.

Post reply on HN