Am I mad or is there a missing ")" on lines and 8 and 9 of the first "C form" that should go before the semicolons?
Claude's Cycles [pdf]
231–240 of 376 posts
Re: Claude's Cycles [pdf]
#232Earlier quoted context omitted.
What is dumb zone?
When the LLMs start compacting they summarize the conversation up to that point using various techniques. Overall a lot of maybe finer points of the work goes missing and can only be retrieved by the LLM being told to search for it explicitly in old logs. Once you compact, you've thrown away a lot of relevant tokens from your problem solving and they do become significantly dumber as a result. If I see a compaction c…
So you use the letter to itself in addition to the compacted context? I am curious what you ask it to include in the letter and how it is different from a custom instruction passed to /compact?
Re: Claude's Cycles [pdf]
#233Earlier quoted context omitted.
The specific sequence of tokens that comprise the Knuth's problem with an answer to it is not in the training data. A naive probability distribution based on counting token sequences that are present in the training data would assign 0 probability to it. The trained network represents extremely non-naive approach to estimating the ground-truth distribution (the distribution that corresponds to what a human brain migh…
>the distribution that corresponds to what a human brain might have produced.. But the human brain (or any other intelligent brain) does not work by generating probability distribution of the next word. Even beings that does not have a language can think and act intelligent.
LLMs also don't work by generating probability distributions of the next word. Your explanation isn't able to explain why they can generate words, let alone sentences.
Re: Claude's Cycles [pdf]
#234[flagged]
Re: Claude's Cycles [pdf]
#235[flagged]
Re: Claude's Cycles [pdf]
#236It's incredible to see work like this from him, at a ripe old age of eighty-six.
Re: Claude's Cycles [pdf]
#237Earlier quoted context omitted.
>the distribution that corresponds to what a human brain might have produced.. But the human brain (or any other intelligent brain) does not work by generating probability distribution of the next word. Even beings that does not have a language can think and act intelligent.
LLMs also don't work by generating probability distributions of the next word. Your explanation isn't able to explain why they can generate words, let alone sentences.
That is exactly how they work.
Re: Claude's Cycles [pdf]
#238It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…
Check out https://unratified.org, it tries to answer that question directly, actually.
Re: Claude's Cycles [pdf]
#239Fascinating report by DEK himself. Time to sit down, read, digest and understand it without the help of LLM.
I don't have time to do that myself yet so I just dug a quick TL;DR rabbit hole for fun: https://ontouchstart.github.io/rabbit-holes/llm_rabbit_hole_...
Lol, it's longer than the original article.
Re: Claude's Cycles [pdf]
#240I wonder how long we have until we start solving some truly hard problems with AI. How long until we throw AI at "connect general relativity and quantum physics", give the AI 6 months and a few data centers, and have it pop out a solution?
I think a very long time because part of our limit is experiment. We need enough experimental results to explain to solve these theoretical mismatches and we don't and at present can't explore that frontier. Once we have more results at that frontier we'd build a theory out from there that has two nearly independent limits for QFT and GR. What we'd be asking if the AI is something that we can't expect a human to solv…
Agreed. We have lots of theories like string theory, but until we can make an experiment to prove one way or another it remains a theory.