Earlier quoted context omitted.
Making mistakes does not make people unintelligent.
People don't hallucinate. That is they can pretty reliably assess if they know or don't know something.
Some people can reliably assess what the know, others cannot
Claude's Cycles [pdf]
341–350 of 376 posts
Re: Claude's Cycles [pdf]
#342Earlier quoted context omitted.
I agree. I met Knuth briefly after a guest lecture at my university a few years ago and although you could tell his body was getting old, his mind was incredibly fresh. Although I'm not as bright as him, I can only hope to be as intellectually curious as him at that age.
I don't even think this is controversial, but I don't think it's at all without causation: not remaining curious, keeping the mind stimulated, etc., accelerates one's decline. If you work in something labour intensive, you should retire young while your body's in good health; if you work in academia you should (strive for emeritus and) never leave! (And if you work in SWE, I don't know, we should probably retire, but…
To put this more succintly I think, the mind loves learning something new. Something to do with new connections in the brain.
Re: Claude's Cycles [pdf]
#343Earlier quoted context omitted.
This is an important point. We know what our DMN is and how we use language as a basis for thought to create concepts and complex ideas. However language also bounds our thought. What about the Dolphin? It is a fundamental philosophical problem of if advanced intelligence can exist without language. We have a pretty good notion that you need some sort of substrate (language) to create intelligence. And we know that m…
The dolphin question, for me, is about what we'd even communicate with a creature that lives in such a different world. Humans mostly live in a 2D environment, for instance - we walk on flat planes, rarely looking up. We always have the ground beneath us, the unattainable sky above. Dolphins live in a 3D space, visiting the air above regularly to breathe, the "ground" below a varying distance away. I have no idea how…
They still have mammalian wet-ware. The dolphin has a relatively advanced neocortex which means they likely have some relatively advanced processing. They also have an interesting part of their brain that we don't have and it is likely for social and emotional information based on their behavior. We suspect they may even have a model of the self.
They still have roughly the same kind of hardware as we do. Their different brain region is kind of like a coprocessor we don't have. But based on their behavior they are likely doing the same things we are. I would say they would be more like an extreme human culture than something alien. They probably have very different category mappings based on echolocation.
I think because we know their brains are doing a lot of things that are analogs to ours, just with different sensory inputs we can reason about a dolphin brain and their semantic concepts and category mappings way easier than an AI. Dolphins do a lot of the same stuff we do. Grief. Social groups. Predicting the future. I would bet at a single level of semantic abstraction we have a lot of concepts that map. They have a lot of the same hormones we do. They react to danger very similar to us. I think a lot maps, we just don't know how to share that with each other beyond observation of one another and offerings like food and things that translate for any mammal.
Re: Claude's Cycles [pdf]
#344Et tu, Knuthus?
(You want the vocative case here, if you're going to shove on a suffix to make it look Latin. The Shakespeare quote is "et tu, Brutè?".)
Re: Claude's Cycles [pdf]
#345Earlier quoted context omitted.
> It’s not impossible, obviously—humans do it It's still not at all obvious to me that LLMs work in the same way as the human brain, beyond a surface level. Obviously the "neurons" in neural nets resemble our brains in a sense, but is the resemblance metaphorical or literal?
I didn’t mean “possible for LLMs”; this is clearly an open question. In fact, I didn’t even mean “possible for a neural network the size of an LLM”. I just meant “possible”.
I'm not actually convinced that computers can replicate what our brains do. I don't know that a turing machine is sufficient for that.
Re: Claude's Cycles [pdf]
#346Earlier quoted context omitted.
Making mistakes does not make people unintelligent.
People don't hallucinate. That is they can pretty reliably assess if they know or don't know something.
Your comment is a perfect example of a human hallucinating something and not knowing they are wrong about it. People are confidently wrong about things _all the time_.
Re: Claude's Cycles [pdf]
#347Earlier quoted context omitted.
> The training data If the prompt is unique, it is not in the training data. True for basically every prompt. So how is this probability calculated?
The prompt is unique but the tokens aren't. Type "owejdpowejdojweodmwepiodnoiwendoinw welidn owindoiwendo nwoeidnweoind oiwnedoin" into ChatGPT and the response is "The text you sent appears to be random or corrupted and doesn’t form a clear question." because the prompt doesnt correlate to training data.
The prompt does correlate to its training data. In this case, since you sent random text, it generated the most likely response to random text.
Re: Claude's Cycles [pdf]
#348Earlier quoted context omitted.
Would you consider someone with anterograde amnesia not to be intelligent?
A very good point. For anyone not familiar with anterograde amnesia, the classical case is patient H.M. ( https://en.wikipedia.org/wiki/Henry_Molaison ), whose condition was researched by Brenda Milner.
And Brenda Milner is still alive at 107. https://en.wikipedia.org/wiki/Brenda_Milner
Re: Claude's Cycles [pdf]
#349Earlier quoted context omitted.
Obviously, a concept (which is an abstraction in more ways than one) is different from a textual representation. But LLMs don't operate on the textual description of a concept when they are doing their thing. A textual description (which is associated with other modalities in the training data) serves as an input format. LLMs perform non-linear transformations of points in their latent space. These transformations an…
> don't operate on the textual description of a concept when they are doing their thing. It could be mapping the text to some other internal representation with connections to mappings from some other text/tokens. But it does not stop text from being the ground truth. It has nothing else going on! The "hallucination" behavior alone should be enough to reject any claims that these are at least minimally similar to ani…
> The "hallucination" behavior alone should be enough to reject any claims that these are at least minimally similar to animal intelligence.
Can you elaborate on why you think this is the case?
Re: Claude's Cycles [pdf]
#350I am not a theoretical CS or math expert by any means, but I have been wrangling coding agents for a while and reading the paper and the problems Stapper had with dealing w/ Claude (context management, instruction following, etc) decided to see if I could replicate with a slightly better harness. The results were pretty interesting: https://github.com/lhl/claudecycles-revisited
- My original setup left traces of the PDF paper and after GPT 5.3-Codex xhigh reached an impasse it went looking for it and found it!
- I went and did cleanroom (basically one-shot) passes for GPT 5.2 xhigh, GPT 5.3-Codex xhigh, and Claude Opus 4.6 ultrathink and 5.2/5.3 found alternate solutions for odd m >= 5 , Opus 4.6 did not find any proofs but tried more approaches to solving.
Full comparison/analysis here: https://github.com/lhl/claudecycles-revisited/blob/main/COMP...
I've also included the session traces and analysis in the repo branches. Also, the AGENTS.md was pretty simple, but that harness produced consistent process outcomes across all three models:
- All built verifiers first
- All maintained worklogs with exact commands
- All archived machine-readable artifacts
- All documented failed approaches
- All maintained restart-safe context capsules