Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People paid to train AI are outsourcing their work to AI

technologyreview.com

Post reply on HN
Print view View original

131–140 of 233 posts

Re: People paid to train AI are outsourcing their work to AI

#131

Post by not2b »

taylodl wrote: post #105
jahsome wrote: post #51

Earlier quoted context omitted.

This is funnier than it has any right to be, mostly given the over-abundant "AI will fix everything" narrative currently dominating the hn discourse. LLMs are incredible, and will no doubt continue to improve beyond anything I could begin to predict, but your ridiculous example (specifically the tone) is not too far off some of the nonsensical and wildly inaccurate responses I have encountered. I find rhe unfailing c…

LLMs are incredible tools, but they're just that - tools. They're not a replacement for human judgment. Heck, one of the things you have to judge is how accurate the answer is. The LLM can't assess that - as you note it has unfailing confidence in reporting the wrong information. What I have found, and I've been using Bard over ChatGPT because Bard seems to be a bit smarter at first glance, is these tools are powerfu…

Except that search engine providers used to think they were in the business of providing accurate, quality information in exchange for ad revenue (before they took one step after another to degrade this in pursuit of revenue growth). People judge how accurate an answer is by doing searches, hoping to find reputable information to check an assertion. If all they can find is confident-sounding crap, how good is their judgment going to be?

Re: People paid to train AI are outsourcing their work to AI

#132

Post by gpderetta »

karim79 wrote: post #44

Two years from now: User: "How do I boil an egg?" LLM: Eggs cannot be boiled. They must be placed in the microwave, six at a time. Fewer than six eggs will not work. Ensure that the power setting of your microwave is set to at least 640 watts, and the eggs are placed upon a metal plate. Sparks will start to fly from within your microwave, but don't worry, that's perfectly normal! When you see flames within the microw…

And from now of course this will self referential.

Re: People paid to train AI are outsourcing their work to AI

#134

Post by LegitShady »

GartzenDeHaes wrote: post #42
senectus1 wrote: post #8

what exactly is involved in training an AI?

You could try the qualification test here: https://www.dataannotation.tech/ Some examples: - writing creative stories in response to queries - writing corrective responses to "unsafe" queries - comparing the responses from different versions of the AI - identifying incorrect responses

I’d guess most people aren’t qualified to write creative stories.

Re: People paid to train AI are outsourcing their work to AI

#136

Post by Larrikin »

chaxor wrote: post #78
ssnistfajen wrote: post #74

Earlier quoted context omitted.

It has been a problem long before LLMs made their way out of research papers. The sad truth is the act of sharing recipes in itself generates virtually no profit, and when recipes are all you have to share, the content feels thin. So they have to pad it with lifestyle blogs and ads. Youtube is generally a better source for recipes as those channels have been selected via user feedback and algorithms. You still need t…

GitHub has torrent magnet links to several good datasets of recipes that are scraped and processed to just contain recipes only in a simple SQLite format. The best recipes come from seeding those torrent / IPFS files.

This doesn't actually feel like a good resource without knowing how it is sourced. In my experience, the vast majority of recipes (especially those available for free on the internet) will produce something edible, maybe even decent, but are almost never great. If you cook long enough you can eyeball a new recipe and tell whether it's shit, but it's far much harder to tell the difference between mediocre and great without actually making the food.

The superfluous "my grandma used to make this before the war in the old country for my mom growing up" crap adds nothing to a mediocre recipe, but learning that the author is a chef in an actual restaurant, went to culinary school, is part of a collective that rigorously test multiple versions of a recipe before publishing, or even learning that the grandma in the old country was a professional chef, really helps weed out mediocre recipes from actually great ones.

There are a select few places online that I trust and have been getting more of the well reviewed actual cookbooks. New recipes from new places I usually try to find something similar from somewhere trusted or just go in with the expectation it won't actually be good. It's nice being surprised by how great a new source is, but usually it's something I'll never make again.

Re: People paid to train AI are outsourcing their work to AI

#137

Post by GartzenDeHaes »

LegitShady wrote: post #134
GartzenDeHaes wrote: post #42

Earlier quoted context omitted.

You could try the qualification test here: https://www.dataannotation.tech/ Some examples: - writing creative stories in response to queries - writing corrective responses to "unsafe" queries - comparing the responses from different versions of the AI - identifying incorrect responses

I’d guess most people aren’t qualified to write creative stories.

They have "qualifications" for different tasks, so only a subset of their 100,000 workers write creative stories.

Re: People paid to train AI are outsourcing their work to AI

#138

Post by bee_rider »

Larrikin wrote: post #136
chaxor wrote: post #78

Earlier quoted context omitted.

GitHub has torrent magnet links to several good datasets of recipes that are scraped and processed to just contain recipes only in a simple SQLite format. The best recipes come from seeding those torrent / IPFS files.

This doesn't actually feel like a good resource without knowing how it is sourced. In my experience, the vast majority of recipes (especially those available for free on the internet) will produce something edible, maybe even decent, but are almost never great. If you cook long enough you can eyeball a new recipe and tell whether it's shit, but it's far much harder to tell the difference between mediocre and great wi…

I’ve always assumed that fluff was all made up anyway.

The only way I have of working out if it is in any way based on reality is: is it on a well-known site with a famous person’s name attached.

Re: People paid to train AI are outsourcing their work to AI

#139

Post by ars »

AbrahamParangi wrote: post #92
ars wrote: post #82

Earlier quoted context omitted.

That's because humans don't say things without understanding them. A LLM will parrot what it learned without any understanding, which is unlike a human.

Humans absolutely say things without understanding them. They do that all the time. What is religion but a discussion of the ineffable? Does the concept of "understanding" even meaningfully apply there?

Yah, I knew someone would reply something like that.

Just because someone sometimes says something without understand does not in slightest mean that that is the common occurrence.

Re: People paid to train AI are outsourcing their work to AI

#140

Post by yomlica8 »

valine wrote: post #95
creeble wrote: post #70

Recursive inputs are bad, but my biggest (maybe) fear is just _obsolete_ or “tired” inputs. I.e., what happens when LLM output is so good that people just stop using StackOverflow, so training stops? Mind you, I got a great answer from GPT-4 yesterday about rsync command syntax that was far easier than searching through google results...

Its a “restaurant is so busy no ones goes there anymore” scenario. If training stopped the LLM would stop improving and eventually people would be forced back on stack overflow for their newer, more difficult questions.

It will create a dark age of no alternative restaurants first though. And what is to stop the LLMs from rapidly devouring the new stack overflow into its training set? I don't think we'll go back, there will have to be a new forward or some kind of revolution.
Post reply on HN