Can LLMs invent better ways to train LLMs?
1–10 of 39 posts
Re: Can LLMs invent better ways to train LLMs?
#2Betteridge's law of headlines: no
Re: Can LLMs invent better ways to train LLMs?
#3Betteridge's law of headlines: no
You'll need the following law too,
No clickbaity article is worth reading.
Re: Can LLMs invent better ways to train LLMs?
#4Re: Can LLMs invent better ways to train LLMs?
#5A better question is "Can LLMs invent anything?"
Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instruction in its code generator, LLMs don't generate previously unseen system responses.
Re: Can LLMs invent better ways to train LLMs?
#6Betteridge's law of headlines: no
The uninformed preliminary answer instead is yes: loaded dice can produce better values.
The article is about using LLMs in an evolutionary framework to design better algorithms for LLM advancement, with particular occasional regard to preference optimization algorithms - and guess what, it seems it worked.
Re: Can LLMs invent better ways to train LLMs?
#7So it does seem to work. That's not clickbait then?
Re: Can LLMs invent better ways to train LLMs?
#8A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
> Can LLMs invent anything
Can they propose a working novelty:
-- after deep thought about idea soundness, probably not at this stage
-- through cycles of trials, not knowing exactly why - probably yes
After all, your hammer needs not intelligence.
Re: Can LLMs invent better ways to train LLMs?
#9A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
What are inventions other than combinations of existing patterns?
What is the human mind if not a computer?
What is the universe if not repeated regurgitations of the four fundamental forces and 12 particles?
Re: Can LLMs invent better ways to train LLMs?
#10A better question is "Can LLMs invent anything ?" Don't misunderstand, building systems models using existing system response as a way of analyzing those systems is a useful methodology and it makes some things otherwise tedious things not so tedious. Much like "high level" languages removed the tedium of writing in assembly code. But for the same reason that a compiler won't emit a new, more powerful, CPU instructio…
What are inventions other than combinations of existing patterns? What is the human mind if not a computer? What is the universe if not repeated regurgitations of the four fundamental forces and 12 particles?
> What are inventions other than combinations of existing patterns?
Well criticized combinations of existing patterns
> What is the human mind if not a computer
A computer with important modules installed
> What is the universe if not repeated regurgitations of the four fundamental forces and 12 particles
Repeated regurgitations which have already produced working structures