Live data from Hacker News

GenAI, the snake eating its own tail

ybrikman.com

101–110 of 138 posts

Re: GenAI, the snake eating its own tail

#101
post #75

Everyone here throwing shade at Stack Overflow is clearly too young to remember the horror of Experts Exchange and every other technical help site prior to SO. For nearly a decade, it was absolutely transformative as a technical help resource. It certainly had its faults, but it was so far ahead of the other options as to be game-changing. I believe that the main reason for SO's decline starting around 2018 was that…

SO was great for a while, then went down the toilet, not because 'eveything had been answered', but because it became a playground of power hungry mods vs resume grinding freshmen patronizing and shutting down 90% of 'normal' users. Even current AI is a 100x better experience than SO ever was. We can all see how post knowledge scarecity and automated contextual niche adaptation reduces exploitation potential for know…

The issue with SO is also that the quality of the answers degrades over time. Asking similar questions will get closed as dupes while the referenced 2011 answer is basically useless nowadays.

Re: GenAI, the snake eating its own tail

#102
post #77
post #75

Everyone here throwing shade at Stack Overflow is clearly too young to remember the horror of Experts Exchange and every other technical help site prior to SO. For nearly a decade, it was absolutely transformative as a technical help resource. It certainly had its faults, but it was so far ahead of the other options as to be game-changing. I believe that the main reason for SO's decline starting around 2018 was that…

Prior to SO we had Usenet, mailing lists, and IRC. They weren't so bad before spammers found them.

Funny enough, you think that irc is dead, like most people, and the spammers... but let me tell you a little secret here:

Re: GenAI, the snake eating its own tail

#103
post #75

Everyone here throwing shade at Stack Overflow is clearly too young to remember the horror of Experts Exchange and every other technical help site prior to SO. For nearly a decade, it was absolutely transformative as a technical help resource. It certainly had its faults, but it was so far ahead of the other options as to be game-changing. I believe that the main reason for SO's decline starting around 2018 was that…

> I believe that the main reason for SO's decline starting around 2018 was that most of the core technical questions had been answered

I believe that the rise of SO was mostly a miracle. A once-in-an-era thing. The evidence is all the other Stack Exchange sites. They all have the same UI and the same moderation model. If SO has some secret sauce they have it too. But most of them were pretty dead and never became an enormous corpus.

Re: GenAI, the snake eating its own tail

#105
post #56
post #48

Earlier quoted context omitted.

No. The idea-expression dichotomy is a common myth about copyright law, right up there with "if I already own the physical cartridge, downloading this game ROM is OK". The ONLY things that matter when determining whether copyright was infringed are "access" and "substantial similarity". The first refers to whether the alleged infringer did, or had a reasonable opportunity to, view the copyrighted work. The second is…

I think what you'll find is that most aren't happy with the current copyright law anyway (I include myself in that group) or don't understand it or don't agree with it, and thus will just shrug. For example, copyright duration is far longer than most people think (life of author plus seventy (or plus ninety-five years if corporation). Corporations treat copyright as a way to create moats for themselves and freeze com…

It's incoherent to be anti-copyright because it's used to freeze out competition by corporations and be pro-AI (which is exactly that, at vastly greater scale).

Re: GenAI, the snake eating its own tail

#106
post #43

Earlier quoted context omitted.

I feel no nostalgia for Stackoverflow. I always found it very frustrating that for a person at the start of the learning curve it was "read only" Actually asking a naive question there was to get horribly flamed on the site. It, and the people using it, were very keen to explain how stupid you were being LLMs on the other hand are sweet and welcoming (to a fault) of the naive newbie I have been learning to use Shell…

It was very strange to me how StackOverflow consistently, every time it was mentioned everyone had exactly the same complaints and nothing changed. They can't have been unaware of the reputation or the sinking activity rates.

>They can't have been unaware of the reputation

Management can be idiots longer than a site can remain solvent.

Re: GenAI, the snake eating its own tail

#107
Foundation model sponsors already pay humans to generate authentic content, especially in technical areas that are underrepresented in general internet scrapes. I would imagine that this trend will continue.

Further, the "model collapse" hypothesis of 2020/2021 seems to have failed to materialize. Maybe we're still too early, and we're not yet seeing negative effects of OpenAI training on OpenAI output. But maybe "slop" is not being rewarded as much as human content, and having humans in the loop (even as readers) is preventing a slide into incoherence.

Will LLMs eventually disincentivize people from producing and publishing new original content? If that content is easily replicated by an LLM query, maybe. And maybe it's not the worst thing in the world. 5 years ago I would have bought an "FFmpeg Cookbook" from O'Reilly, but now I would just tell Claude exactly what I'm trying to achieve. As a consumer, I'm better off, and arguably we've saved the author of a hypothetical FFmpeg Cookbook weeks out of their precious life. Weeks they could spend doing something—anything—more valuable than rewording FFmpeg documentation.

Re: GenAI, the snake eating its own tail

#108
post #83

Earlier quoted context omitted.

Eventually it won’t matter. People will forget what it was like to have authentic content. They’d look at it like how we might view monks in the middle ages copying the bible a hundred times by hand. Even today people don’t even trust themselves to write an email. You see people on HN and reddit openly admitting they used ai to help make their post because they believe they cannot write. The march to illiteracy and i…

This is a plain fact. The amount of obviously GPT’d text out there is so breathtaking. The good news is that all of this shit people are squeezing out of GPT is really reinvigorating my desire to work on novel creative projects. In nearly every form of entertainment over the past 2 years, quality has degraded rapidly, to the point where most Reddit threads really aren’t worth reading - they’re all chock full of It’s…

>In nearly every form of entertainment over the past 2 years

I mean, it really was on a huge downward trend before two years ago, and you hit on much of that.

Social media and the online advertising age had really destroyed a lot of entertainment well before AI was an issue. Bot-like humans would just copy and mish-mash existing media poorly and attempt to gather the ad dollars for themselves.

Honestly a lot of the enjoyable content I watch these days is from individuals/groups that aren't chasing ad views, but doing more 'donation begging' from their own audience which means they must maintain some level of quality for continued patronage.

One particular problem we seem to have is that we see AI as 'the problem' and not just one issue in a problematic system. Why doesn't Reddit do anything about crap posts and content? They make money off of convincing advertisers that lots of views happen on their site and they should sell ads. Why doesn't Google improve their services? They are a near monopoly in online advertising and service improvement for the end user doesn't increase their revenue.

It's just a system where one group is trying to extract wealth from Google/Facebook whereas Google/Facebook have nearly perfected extracting wealth from you.

Re: GenAI, the snake eating its own tail

#109

> he took a PDF of my book Terraform: Up & Running, uploaded it into a GenAI tool, and asked the tool to follow the guidance in the book to generate Terraform code This is ridiculous - AI doesn't need to be fed a PDF of a Terraform book to know how to Terraform. Blowing out context with hundreds of OCR'd pages of generic text on how to terraform isn't going to help anything. The model that is broken is really ultimat…

>I'm not talking about video game artists

You are talking about some video game artists, and while not their fault directly, EA pushing out SportsBall 2026 for the 40th year in a row is just a soulless corporate money printing machine.

Re: GenAI, the snake eating its own tail

#110
post #5

Earlier quoted context omitted.

>If the argument is sustainability of training, I'm skeptical we need these payment models. That seems to be the argument: LLM adoption leads to drop of organic training data, leading LLMs to eventually plateau, and we'll be left without the user-generated content we relied on for a while (like SO) and with subpar LLM. That's what I'm getting from the article anyway.

There are so many things wrong with the points this article repeats, but those are soundbites at this point so I'm not sure one can even argue against them anymore. Still, for the one about organic data (or "pre-war steel") drying out, it's not a threat to model development at all. People repeating this point don't realize that we already have way more data than we need . We got to where we are by brute-forcing the p…

>We got to where we are by brute-forcing the problem

This has been a bit of a concern of mine. That we have to do things the hard way for a long time, and in doing so make a massive amount of fast hardware. Then we get some breakthru that massively drops the amount of compute necessary, the surplus we suddenly have may lead to some kind of AI capability explosion.

Post reply on HN