I'm sure LLMs can optimize the training of other LLMs (either by inventing new ways or fine tuning existing ones). But we can't predict whether this will result in a giant's leap in the field, or just small increments. That's the definition of singularity, isn't it?
Can LLMs invent better ways to train LLMs?
11–20 of 39 posts
Re: Can LLMs invent better ways to train LLMs?
#12can LLMs optimize anything?
Re: Can LLMs invent better ways to train LLMs?
#13Of course, they can invent anything. A better question is how efficient? Because even with brute force you can invent anything: https://libraryofbabel.info/
Re: Can LLMs invent better ways to train LLMs?
#14can LLMs optimize anything?
They optimised my coding speed for sure.
Re: Can LLMs invent better ways to train LLMs?
#15A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
can you propose a concrete prediction here?
Re: Can LLMs invent better ways to train LLMs?
#16A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
I think it's probably unreasonable to expect them to without giving them the ability to experiment and test ideas.
Pretty much no inventions were invented just by thinking, which is the environment most LLMs have.
Re: Can LLMs invent better ways to train LLMs?
#17They are very useful when ideating with a human. On their own they could veer off into uncertain territory, and likely make mistakes obvious to humans.
Re: Can LLMs invent better ways to train LLMs?
#18A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
I think it's probably unreasonable to expect them to without giving them the ability to experiment and test ideas. Pretty much no inventions were invented just by thinking, which is the environment most LLMs have.
I think this is the key point. There is no feedback loop right now.
Re: Can LLMs invent better ways to train LLMs?
#19A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
[deleted]
Re: Can LLMs invent better ways to train LLMs?
#20So it does seem to work. That's not clickbait then?
[deleted]