Live data from Hacker News

AI models collapse when trained on recursively generated data

nature.com

41–50 of 212 posts

Re: AI models collapse when trained on recursively generated data

#41

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#42

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#43

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#44

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#45

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#46

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#47

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#48

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#49

This seems extremely interesting, but I don't have the time right now to read this in depth (given I would also need to teach myself a bunch of technical concepts too). Anyone willing to weigh in with a theoretical intuition ? The one in the paper is just a little inaccessible to me right now.

[deleted]

Re: AI models collapse when trained on recursively generated data

#50
post #24
post #15

Meanwhile OpenAI, Anthropics, trains on AI generated data to improve their models, and it works. https://openai.com/index/prover-verifier-games-improve-legib... https://www.anthropic.com/research/claude-character

Cheese and Chalk. It is very different to generate synthetic datasets to assist in targeted training , vs ingesting LLM output from web scraping.

Yes -- said another way, if you're an ML researcher and you have human-provided (scraped) data, and an ability to generate synthetic data, then until recently, you had a controllable parameter: how much of your training data for your new model should be synthetic? You can vary this, run multiple experiments, and choose how much synthetic data to use -- and you can vary the specific configs about how that synthetic data is generated.

If synthetic data is mixed into your upstream data sources in a way you cannot control, then your ML team loses a valuable controllable parameter.

Post reply on HN