Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GenAI, the snake eating its own tail

ybrikman.com

Post reply on HN
Print view View original

11–20 of 138 posts

Re: GenAI, the snake eating its own tail

#11

Post by mapontosevenths »

lbrito wrote: post #5
mrcwinn wrote: post #2

Pay per crawl of StackOverflow wouldn't encourage me to post more on StackOverflow. (Not that I was anyway.) Presumably you'd need to pay content creators, but that seems quite inefficient: 1. I pay OpenAI 2. OpenAI rev shares to StackOverflow 3. StackOverflow mostly keeps that money, but shares some with me for posting 4. I get some money back to help pay OpenAI? This is nonsense. And if the frontier labs are right…

>If the argument is sustainability of training, I'm skeptical we need these payment models. That seems to be the argument: LLM adoption leads to drop of organic training data, leading LLMs to eventually plateau, and we'll be left without the user-generated content we relied on for a while (like SO) and with subpar LLM. That's what I'm getting from the article anyway.

The article gets the part about organic data dying off right. Look at Google SERP's for an example. Almost nobody clicks through to the source anymore, so ad revenue is drying up for them and people are publishing less or publishing in places that pay them directly and live behind a paywall like Medium. Which means Google has less data to work with.

That said, what it misses is that the AI prompts themselves become a giant source of data. None of these companies are promising not to use your data, and even if you don't opt-in the person you sent the document/email/whatever to will because they want it paraphrased or need help understanding it.

Re: GenAI, the snake eating its own tail

#13

Post by alfalfasprout »

znsksjjs wrote: post #3

We’ve seen decades of growing wage gaps and erosion of labors strength. The current elites don’t really care to enrich the people. Why would they care to do anything about this problem? They likely don’t see it as a problem at all. If they did actually stumble on AGI (assuming it didn’t eat them too) it would be used by a select few to enslave or remove the rest of us.

Not sure why this is being downvoted. It's spot on. You see folks like Dario et al. raising the alarm bells about what they claim is coming... while working as hard as they can to bring that gloomy future to fruition.

No one in power is going to help unless there's money in it.

Re: GenAI, the snake eating its own tail

#14

Post by logifail »

furyofantares wrote: post #4

The article feels very confused to me. Example 1 is bad, StackOverflow had clearly plateaued and was well into the downward freefall by the time ChatGPT was released. Example 2 is apparently "open source" but it's actually just Tailwind which unfortunately had a very susceptible business model. And I don't really think the framing here that it's eating its own tail makes sense. It's also confusing to me why they're t…

> They can try to solve that problem

Well, they could always try actually paying content creators. Unlike - for instance - StackOverflow.

Re: GenAI, the snake eating its own tail

#15

Post by alfalfasprout »

jaredcwhite wrote: post #6

“We can’t put the genie back in the bottle.” Actually we can. And we will.

Agreed, it's funny how people have taken unrestrained use of AI as an axiom at this point. There very much is still time to significantly control it + regulate it. Is there enough appetite by those in power (across the political spectrum)? Right now I don't think so.

Re: GenAI, the snake eating its own tail

#16

Post by thatguy0900 »

alfalfasprout wrote: post #13
znsksjjs wrote: post #3

We’ve seen decades of growing wage gaps and erosion of labors strength. The current elites don’t really care to enrich the people. Why would they care to do anything about this problem? They likely don’t see it as a problem at all. If they did actually stumble on AGI (assuming it didn’t eat them too) it would be used by a select few to enslave or remove the rest of us.

Not sure why this is being downvoted. It's spot on. You see folks like Dario et al. raising the alarm bells about what they claim is coming... while working as hard as they can to bring that gloomy future to fruition. No one in power is going to help unless there's money in it.

You can also see all of these people building survival bunkers.

Re: GenAI, the snake eating its own tail

#17

Post by zzzeek »

mrcwinn wrote: post #2

Pay per crawl of StackOverflow wouldn't encourage me to post more on StackOverflow. (Not that I was anyway.) Presumably you'd need to pay content creators, but that seems quite inefficient: 1. I pay OpenAI 2. OpenAI rev shares to StackOverflow 3. StackOverflow mostly keeps that money, but shares some with me for posting 4. I get some money back to help pay OpenAI? This is nonsense. And if the frontier labs are right…

> If the argument is sustainability of training,

that is the argument, yes.

Claude clearly got an enormous amount of its content from Stackoverflow. Which has mostly ceased to be a source of new content. However unlike the author I dont see any way to fix this; stackoverflow was only there because people had technical questions that needed answers.

Maybe if the LLMs do indeed start going stale as there's not enough training data for new technologies, Q&A sites like Stackoverflow would still have a place, since people would still resort to asking each other questions rather than LLMs that dont have training data for a newer technology.

Re: GenAI, the snake eating its own tail

#18

Post by lbrito »

alfalfasprout wrote: post #13
znsksjjs wrote: post #3

We’ve seen decades of growing wage gaps and erosion of labors strength. The current elites don’t really care to enrich the people. Why would they care to do anything about this problem? They likely don’t see it as a problem at all. If they did actually stumble on AGI (assuming it didn’t eat them too) it would be used by a select few to enslave or remove the rest of us.

Not sure why this is being downvoted. It's spot on. You see folks like Dario et al. raising the alarm bells about what they claim is coming... while working as hard as they can to bring that gloomy future to fruition. No one in power is going to help unless there's money in it.

Its being downvoted because HN has a very active billionaire-techbro-fanbase.

Also who's this Dario?

Re: GenAI, the snake eating its own tail

#19

Post by jaredcwhite »

mapontosevenths wrote: post #12
semiquaver wrote: post #8

Earlier quoted context omitted.

How?

It was presented without explanation and can be ignored without explanation.

You need an explanation of how people make norms & laws regarding what is acceptable or unacceptable in society and industry?

Re: GenAI, the snake eating its own tail

#20

Post by mapontosevenths »

npinsker wrote: post #7
furyofantares wrote: post #4

The article feels very confused to me. Example 1 is bad, StackOverflow had clearly plateaued and was well into the downward freefall by the time ChatGPT was released. Example 2 is apparently "open source" but it's actually just Tailwind which unfortunately had a very susceptible business model. And I don't really think the framing here that it's eating its own tail makes sense. It's also confusing to me why they're t…

“Well, Reddit is growing, which contradicts my point, but I really feel like it’s not”

Reddit is growing because they introduced automatic machine translation and Indians have been joining at an increasing rate. That content is mixed into the English language content, but is of very low quality and irrelevant to many native English speakers. Similarly they mix the English content in with the Indian content.

Essentially, Reddit is also eating it's own tail to survive as the flood of low quality irrelevant content is making the platform worse for speakers of all languages but nobody cares because "line go up."

Post reply on HN