AI models collapse when trained on recursively generated data
1–10 of 212 posts
Re: AI models collapse when trained on recursively generated data
#2Which is good background to this story about Reddit locking down robots.txt and trying to get money from the AI teams scraping their content.
Re: AI models collapse when trained on recursively generated data
#3It’s like the ai generated version of index funds.
Re: AI models collapse when trained on recursively generated data
#4Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B rejects, both parties get nothing. Standard economic theory suggests A can offer $1 and B will accept, because otherwise B gets nothing. Spoiler for those of you who haven't seen how standard economic theory plays out in real life, that's not how the game went - typically, offers below ~$30 or so got rejected, because B was a real feeling person who felt like they were getting screwed and opted to punish A for doing so. The exception to this - the people who would take the $1 offer - were people who had been taught economic theory. It turns out you _could_ screw them over and they'd pat themselves on the backs for being very wise.
The "tragedy of the commons" is another one of those parts of standard economic theory that never actually played out in reality - we've got examples from all over the world of communities implementing practices and often entire belief systems that led them to be responsible stewards of shared resources without requiring unilateral ownership of that resource and singular acquisition of the benefits of that stewardship, and yet first on the lips of every modern capitalist when describing why they're at a disadvantage if they're not the ones polluting the water supply is the tragedy of the commons.
Re: AI models collapse when trained on recursively generated data
#5Seems analogous to the effect of echo chambers on humans
Re: AI models collapse when trained on recursively generated data
#6Which is good background to this story about Reddit locking down robots.txt and trying to get money from the AI teams scraping their content. https://news.ycombinator.com/item?id=41057033
If they're considering Reddit content to be free of generated material, I've got bad news for them. It's not quite the Chernobyl-grade hole that Pinterest has become, but it's hardly "low background".
Re: AI models collapse when trained on recursively generated data
#7This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too).
Anyone willing to weigh in with a theoretical intuition? The one in the paper is just a little inaccessible to me right now.
Re: AI models collapse when trained on recursively generated data
#8As far as I understand Douglas Hofstadter's Godel, Escher, Bach - self-referential recursive structures (strange loops) are the foundation of consciousness (among other interesting things). I've been watching to see if LLM's becoming self-referential actually improves them as opposed to degrades them.
Re: AI models collapse when trained on recursively generated data
#9A lot of these papers are wrong. They do something wrong in their setup and then claim their conclusion shows show general truth.
Publishing in nature in ML can actually be a red flag, because they're really not well equipped to evaluate a lot of claims.
The latest llama model got a lot of its data using labels from llama2, and every frontier lab is talking about self training as the future.
Re: AI models collapse when trained on recursively generated data
#10Seems analogous to the effect of echo chambers on humans
Or navel-gazing. In fact, that's one of the classically known flaws. (So well known that it has many names: ivory tower, navel gazing, getting stuck in your own head...)
If you don't compare your thoughts to the outside world, it's easy for them to diverge more and more from reality.