Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI models collapse when trained on recursively generated data

nature.com

Post reply on HN
Print view View original

31–40 of 212 posts

Re: AI models collapse when trained on recursively generated data

#31

Post by renewiltord »

roughly wrote: post #4

Back when I was getting my econ degree, we were taught about the Ultimatum game, which goes like this: You get two participants who don't know each other and will (ostensibly) never see each other again. You give one of them $100, and they make an offer of some portion of it to the other. If the other accepts, both parties keep their portion - so, if A offers B $20, and B accepts, A keeps $80 and B keeps $20, if B re…

If you iterate the game, it’s obvious. I, as the responder, control the proposer’s income. Extend to infinity with knowledge of iteration and you reach symmetry between proposer and responder.

Re: AI models collapse when trained on recursively generated data

#32

Post by autokad »

ainoobler wrote: post #28
echelon wrote: post #22

Earlier quoted context omitted.

I'm long on synthetic data. If you think about evolution and hill climbing, of course it works. You have a pool of information and you accumulate new rearrangements of that information. Fitness selects for the best features within the new pool of data (For primates, opposable thumbs. For AI art, hands that aren't deformed.) It will naturally drift to better optima. RLHF, synthetic data, and enrichment are all we need…

Are you sure about this? It's well known that cannibalism in animals leads to degenerative disorders.

I think the direct action of a person taking their idea and thoughts and going through it many times (making changes / updates / fixes) fits better than eating something. however, I do think you still some form of validation data to ensure these are good changes.

However, I do get the spirit of the article, that as more information generated online is done by LLms, the validity and use of the output decreases

Re: AI models collapse when trained on recursively generated data

#33

Post by ainoobler »

autokad wrote: post #32
ainoobler wrote: post #28

Earlier quoted context omitted.

Are you sure about this? It's well known that cannibalism in animals leads to degenerative disorders.

I think the direct action of a person taking their idea and thoughts and going through it many times (making changes / updates / fixes) fits better than eating something. however, I do think you still some form of validation data to ensure these are good changes. However, I do get the spirit of the article, that as more information generated online is done by LLms, the validity and use of the output decreases

What exactly is doing the validation?

Re: AI models collapse when trained on recursively generated data

#34

Post by hiddencost »

slashdave wrote: post #21
hiddencost wrote: post #9

A lot of these papers are wrong. They do something wrong in their setup and then claim their conclusion shows show general truth. Publishing in nature in ML can actually be a red flag, because they're really not well equipped to evaluate a lot of claims. The latest llama model got a lot of its data using labels from llama2, and every frontier lab is talking about self training as the future.

Who are "they"? And do you actually believe the practice of publishing unvetted preprints is a good thing in ML research?

Non sequitur? I never said that.

Good venues include main track NeurIPS, ICML, ACL, e.g.

Nature is notorious for publishing PR pieces that don't reproduce, and their ML theory publishing has been quite poor. They do pretty well on things like AlphaGo, materials science, or weather modeling because it's more in their wheelhouse and the results don't require a deep understanding of info theory or ML practice.

Re: AI models collapse when trained on recursively generated data

#35

Post by [deleted] »

padraicmahoney wrote: post #7

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#36

Post by [deleted] »

padraicmahoney wrote: post #7

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#37

Post by [deleted] »

padraicmahoney wrote: post #7

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#38

Post by [deleted] »

padraicmahoney wrote: post #7

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#39

Post by [deleted] »

padraicmahoney wrote: post #7

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#40

Post by [deleted] »

padraicmahoney wrote: post #7

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]
Post reply on HN