I can see a high value startup, that will provide Human Intelligence with real Humans, locked in the room, with no network, books, LLMs and monitored 24x7 with cameras.
[deleted]
Amazon will stop accepting new customers for Mechanical Turk
61–70 of 72 posts
Re: Amazon will stop accepting new customers for Mechanical Turk
#62> In a snake-eating-its-own-tail irony, a 2023 analysis found that between 33% and 46% of workers on the platform were using large language models to complete their tasks, I assume AI use by workers has risen to the point where it renders Mechanical Turk pointless.
I don't see why I would care how they do the job. Just do the job, I have other things to do.
If you get someone to mow your lawn, do you not care if (a) they use an automated machine that you could rent by the hour for far less, and/or (b) at the end your lawn is actually in a good shape?
Re: Amazon will stop accepting new customers for Mechanical Turk
#63Earlier quoted context omitted.
"Artificial AI" and "Artificial Artificial intelligence" are the same thing. Come on. This site can do better.
yes, this site can do a lot better than nitpicking acronyms
You're the one wasting time with another incorrect comment.
This point is that AAAI and AAI are not the same thing.
Re: Amazon will stop accepting new customers for Mechanical Turk
#64Re: Amazon will stop accepting new customers for Mechanical Turk
#65> In a snake-eating-its-own-tail irony, a 2023 analysis found that between 33% and 46% of workers on the platform were using large language models to complete their tasks, I assume AI use by workers has risen to the point where it renders Mechanical Turk pointless.
This likely means those consuming the outputs of Mechanical Turk don't have a good way to measure the value (aka quality) of the outputs. If they did - then they shouldn't care whether it's a human or a LLM. And if it's a LLM - then the cost will roughly correlate to the MIN(cost of the LLM, cost of a human) to do the task.
> then they shouldn't care whether it's a human or a LLM.
I imagine that the whole point of posting a task to Mechanical Turk nowadays is that you want it to be completed by humans. Either because you are after the small discrepancy between AI and human performance, or because humans are the object of your investigation.
Re: Amazon will stop accepting new customers for Mechanical Turk
#66Earlier quoted context omitted.
I don't see why I would care how they do the job. Just do the job, I have other things to do.
If you get someone to mow your lawn, do you not care if (a) they use an automated machine that you could rent by the hour for far less, and/or (b) at the end your lawn is actually in a good shape?
I have very low expectations for $2 per hour. If I would feel the need to use such service I would expect them to cut all corners. If you have a subscription for a decent LLM, know how to get my data into it and know how to write a prompt that does the job you might have 3 skills I don't or I might not have time to do it.
If the lawn looks great you get to do it again. I could ask the guy you send how much you are paying him and I could ask you how much the mower costs but it wouldn't necessarily be a good use of my time.
Re: Amazon will stop accepting new customers for Mechanical Turk
#67Earlier quoted context omitted.
If you get someone to mow your lawn, do you not care if (a) they use an automated machine that you could rent by the hour for far less, and/or (b) at the end your lawn is actually in a good shape?
I have very low expectations for $2 per hour. If I would feel the need to use such service I would expect them to cut all corners. If you have a subscription for a decent LLM, know how to get my data into it and know how to write a prompt that does the job you might have 3 skills I don't or I might not have time to do it. If the lawn looks great you get to do it again. I could ask the guy you send how much you are pa…
Say you notice they come in the morning, put a small machine on the grass, hit a button, and leave and come back 4h later to pick up the machine and charge you - wouldn't you be at least curious to see how much this machine costs? Or if there was another company that would lease such machines for much less than what you paid per hour, wouldn't you possibly prefer to contract them, even if you may have to pick up and drop off the auto lawn mower yourself?
And conversely, say that the person or machine that you paid to mow the lawn instead drenches your yard in herbicide and then salts the earth. Or vice versa, it starts planting weed seeds, or brings a bunch of ticks in the grass etc. You would probably not be happy to let this process finish and then say "oh well, guess I won't be hiring them again!" and just move on.
Re: Amazon will stop accepting new customers for Mechanical Turk
#68Earlier quoted context omitted.
I have very low expectations for $2 per hour. If I would feel the need to use such service I would expect them to cut all corners. If you have a subscription for a decent LLM, know how to get my data into it and know how to write a prompt that does the job you might have 3 skills I don't or I might not have time to do it. If the lawn looks great you get to do it again. I could ask the guy you send how much you are pa…
Say you notice they come in the morning, put a small machine on the grass, hit a button, and leave and come back 4h later to pick up the machine and charge you - wouldn't you be at least curious to see how much this machine costs? Or if there was another company that would lease such machines for much less than what you paid per hour, wouldn't you possibly prefer to contract them, even if you may have to pick up and…
The whole point of delegation is to free up time to do something else. There is a store here where the owner always runs around like a madman. I ask him why he doesn't have his employees do all those things (they mostly stand idle chatting) He said: Then I have to explain everything and they will probably do it wrong.
If I'm getting a $200 mowing bill it could be worth looking at the machines. If I pay you $4 per week I wouldn't worry about it.
Re: Amazon will stop accepting new customers for Mechanical Turk
#69> In a snake-eating-its-own-tail irony, a 2023 analysis found that between 33% and 46% of workers on the platform were using large language models to complete their tasks, I assume AI use by workers has risen to the point where it renders Mechanical Turk pointless.
I guess that’s the dead internet theory. People are trying to train llms using mt, but just training on itself.
Re: Amazon will stop accepting new customers for Mechanical Turk
#70> In a snake-eating-its-own-tail irony, a 2023 analysis found that between 33% and 46% of workers on the platform were using large language models to complete their tasks, I assume AI use by workers has risen to the point where it renders Mechanical Turk pointless.
I don't see why I would care how they do the job. Just do the job, I have other things to do.
If you want data to train a Llm, you might want human data for specific parts vs Llm.