Fascinating, does anyone have any good books on this?
Learning Theory from First Principles [pdf]
61–66 of 66 posts
Re: Learning Theory from First Principles [pdf]
#62Earlier quoted context omitted.
I've never heard of an LLM making up a new idea. Shouldn't this only work if your thing has already been tried before?
Have you heard of a person making up a new idea? Can you definitely state that it's not just a combination of a few previous ideas?
The brain is able to make better associations than an LLM. From what I’ve seen, LLMs always stay on topic, even when that means outputting only platitudes. A human can go past that and try different approaches to a question.
It can also take part of a problem, choose a lead at random, think through the results, and step back if that doesn’t work. A forward pass in an LLM doesn’t do that, yet.
Basically it seems to me that LLMs may have part of what makes us good at inventing ideas, but they’re missing part of that process.
Re: Learning Theory from First Principles [pdf]
#63Earlier quoted context omitted.
Have you heard of a person making up a new idea? Can you definitely state that it's not just a combination of a few previous ideas?
The brain is able to make better associations than an LLM. From what I’ve seen, LLMs always stay on topic, even when that means outputting only platitudes. A human can go past that and try different approaches to a question. It can also take part of a problem, choose a lead at random, think through the results, and step back if that doesn’t work. A forward pass in an LLM doesn’t do that, yet. Basically it seems to me…
I don't think this is a fundamental limitation. If the LLM is trained (through RLHF or something else) to go on a chain-of-internal-monologue (which may take arbitrarily long) to figure out what it will answer, then this kind of adaptive amount of compute can be achieved.
Re: Learning Theory from First Principles [pdf]
#64Earlier quoted context omitted.
The brain is able to make better associations than an LLM. From what I’ve seen, LLMs always stay on topic, even when that means outputting only platitudes. A human can go past that and try different approaches to a question. It can also take part of a problem, choose a lead at random, think through the results, and step back if that doesn’t work. A forward pass in an LLM doesn’t do that, yet. Basically it seems to me…
I don't think this is a fundamental limitation. If the LLM is trained (through RLHF or something else) to go on a chain-of-internal-monologue (which may take arbitrarily long) to figure out what it will answer, then this kind of adaptive amount of compute can be achieved.
Sure, but then how good are Transformers at making connections between distant concepts? The brain seems way better at this, and it doesn't feel like conversation.
Re: Learning Theory from First Principles [pdf]
#65Earlier quoted context omitted.
I don't think this is a fundamental limitation. If the LLM is trained (through RLHF or something else) to go on a chain-of-internal-monologue (which may take arbitrarily long) to figure out what it will answer, then this kind of adaptive amount of compute can be achieved.
Sure, but then how good are Transformers at making connections between distant concepts? The brain seems way better at this, and it doesn't feel like conversation.
One of the big hopes of this "train on all text in existence" (including academic works and everything else) is that this enables connections that no human could make before, since there probably a total of zero people who are specialists in medieval history and molecular biology and also music theory and quantum computational complexity and what have you. However current models seem not to do much of this in my experience. Typically they remain in the specialist "persona" that rarely brings up connections that would be known by other human specialists (whom it would otherwise have no issue impersonating). But I'd imagine this can also be improved somehow.
Think about how often narrowly specialized academics reinvent a bad version of something that already has an elaborately understood theory in another field. (a meme example is when a medical researcher working on insulin metabolism reinvented the trapezoid rule for integration - but of course many more mundane cases happen every day). It would be a great opportunity to avoid this by checking with an LLM. Arguably if that researcher fed his idea to an LLM as it exists today, the LLM would have recognized that this is just the trapezoid rule. In fact I also use it for such "sounding board" where it gives me good googleable terms like, "what you're looking for is called an XYZ".
Re: Learning Theory from First Principles [pdf]
#66Earlier quoted context omitted.
Could you expand or provide a link to a good resource for me to understand this? If the judge program should say terminates yes/no and the program given is `while True: continue`, I guess the argument is that in the finite case, you could in principle just enumerate all programs that don't terminate and identify them as such?
In principle, you can enumerate all possible memory states of the system and determine what the next memory state would be from each one (including multiple possible next states if you account for things like interrupts) Then you treeshake the unreachable parts of that directed graph from the start state, and look for closed loops in what remains.
Makes sense! Thanks!