The article contains no proof of theorem 3.1 and finding counterexamples seems trivial. Adult male weight can be modeled by N(85, 20). You can recursively "train" the model on data it generates without having it collapse. It will stay stationary as long as the samples are large enough.
AI models collapse when trained on recursively generated data
61–70 of 212 posts
Re: AI models collapse when trained on recursively generated data
#62Data in --> slop out.
Slop in --> yikes
Re: AI models collapse when trained on recursively generated data
#63Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B re…
[deleted]
Re: AI models collapse when trained on recursively generated data
#64Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B re…
> It turns out you _could_ screw them over and they'd
End up with a dollar in their pocket which they otherwise wouldn't have.
The Ultimatum game is a useful insight into human psychology: for one thing, it tells us who thinks that the defector in this equilibrium is better off than a counterfactual cooperator.
Ah, but they have their pride! Ok. My pride is not affected by someone else having 99 bucks they didn't earn, and myself $1 likewise. Maybe that other fellow really needed the money.
Re: AI models collapse when trained on recursively generated data
#65Earlier quoted context omitted.
I think the direct action of a person taking their idea and thoughts and going through it many times (making changes / updates / fixes) fits better than eating something. however, I do think you still some form of validation data to ensure these are good changes. However, I do get the spirit of the article, that as more information generated online is done by LLms, the validity and use of the output decreases
What exactly is doing the validation?
depends on what one was doing. could be as simple as re-writing a sentence and asking someone if it looks better
Re: AI models collapse when trained on recursively generated data
#66Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B re…
You may be interested in some of the foundational papers exploring game theory models similar to the Ultimatum game[1][2]. These are known as Iterated Prisoner's Dilemmas.
---
[1] The Evolution of Cooperation (https://ee.stanford.edu/~hellman/Breakthrough/book/pdfs/axel...)
[2] Evolutionary Dynamics of Spatial Games (https://www.sciencedirect.com/science/article/abs/pii/016727...)
Re: AI models collapse when trained on recursively generated data
#67Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B re…
This reminds me of Lord of the Flies. The real version of the events turned out very differently. https://www.newsweek.com/real-lord-flies-true-story-boys-isl...
Rebecca Solnit wrote a book, "A Paradise Built in Hell", on how people behave during disasters, and found broadly the same thing - contra the prepper myths, most people most of the time faced with disaster come together to work cooperatively to help each other.
We're a fundamentally social species - we've got smaller brains than Neanderthals did, we're not a particularly tough species, but we're very, very good at cooperating with each other.
Re: AI models collapse when trained on recursively generated data
#68Earlier quoted context omitted.
> If you think about evolution and hill climbing, of course it works. You don't even need to go that far. How do most children learn? By reading textbooks and listening to lesson plans assembled by their teachers from all the relevant content the teachers have experienced. Our education systems are built on synthetic data that is created for optimized learning, so that every child doesn't have to prove the universe f…
That isn't synthetic data in any reasonable or meaningful sense of the term. You could describe a textbook as a synthesis, sure, in a sense which absolutely does not track with the 'synthetic' in 'synthetic data'. Unless the textbook is AI-generated, and I expect that in 2024, the number of AI-generated textbooks is not zero.
It’s an analogy. The learning materials teachers create for students is very much like synthetic data, it’s just not assembled from algorithmic output.
Re: AI models collapse when trained on recursively generated data
#69Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B re…
If you iterate the game, it’s obvious. I, as the responder, control the proposer’s income. Extend to infinity with knowledge of iteration and you reach symmetry between proposer and responder.
> If you iterate the game, it’s obvious.
We're shockingly bad at doing this in modern society. Our temporal planning horizon is somewhere between 6 months and 5 years, whereas our lifespans are around 75-80.
Re: AI models collapse when trained on recursively generated data
#70Meanwhile OpenAI, Anthropics, trains on AI generated data to improve their models, and it works. https://openai.com/index/prover-verifier-games-improve-legib... https://www.anthropic.com/research/claude-character
I'm long on synthetic data. If you think about evolution and hill climbing, of course it works. You have a pool of information and you accumulate new rearrangements of that information. Fitness selects for the best features within the new pool of data (For primates, opposable thumbs. For AI art, hands that aren't deformed.) It will naturally drift to better optima. RLHF, synthetic data, and enrichment are all we need…
Data created automatically is not the same as human curated data, though both are synthetic. Auto-created data often suffers from a host of demerits (duplication, bias, error, unnatural distribution, irrelevance to learn the intended domain, etc, etc). Human curated data usually avoids these pitfalls, and thus is far more valuable when training -- otherwise all human teachers would be equally good. So auto- vs curated- data are incomparable when training naive neophytes like ML models, or children.