Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

311–320 of 376 posts

Re: Claude's Cycles [pdf]

#311

Earlier quoted context omitted.

Sure, if you want to speak with the precision of a sledgehammer instead of a scalpel

All that needed to be conveyed was that there are humans who cannot create new memories. That is enough to pose the philosophical question about these models having intelligence. Anything more is just adding an anecdote that isn't necessary.

Why would adding more information and context be unnecessary? And why is that bad?

Re: Claude's Cycles [pdf]

#312

Earlier quoted context omitted.

I think you can look at it dispassionately from a systems perspective. There is not /really/ a quantifiable threshold for capital I Intelligence. But there is a pretty well agreed set of properties for biological intelligence. As humans, we have conveniently made those properties match things only we have. But you can still mechanistically separate out the various parts of our brain, what they do, and how they intera…

> The other thing is, human intelligence is the only real intelligence we know about. There's a long and proud history of discounting animal intelligence, probably because if we actually thought animals were intelligent we'd want to stop eating them. Octopodes are sentient. Cetaceans have well-developed language. Elephants grieve their dead. Anyone who has owned a dog knows that it has some intelligence and is capabl…

This is an important point. We know what our DMN is and how we use language as a basis for thought to create concepts and complex ideas. However language also bounds our thought. What about the Dolphin? It is a fundamental philosophical problem of if advanced intelligence can exist without language. We have a pretty good notion that you need some sort of substrate (language) to create intelligence. And we know that mapping the internal state of a brain from inside of itself is incredibly hard and the way our human brain evolved to do it is really fascinating but also full of hacks and mismatched mappings based on what we know is actually going on.

Cognitive computer science explores this whole area of mapping language and the underlying semantic meaning. Ultimately, these intelligences will be bound by physics (unless some new physics or understanding therein happens). And classical intelligences are still bound by classical physics. So I am not sure we can't relate to these other intelligences. We may be limited to some translation layer that does not fully map, but can we still relate to some other consciousness? For that matter consciousness is just another word that vaguely maps to a vast and extremely complex thing in the human brain and each person has a different understanding of what that is. I don't really have any conclusions, you brought up interesting points. We should sit within this realm of inquiry with a lot of humility IMO.

Re: Claude's Cycles [pdf]

#313
post #305

I was a bit interested to do a replication and see if better harness could avoid some of the problems they ran w/ context management, poor instruction following, etc and it looks like yes, it's definitely possible. Here's my repo: https://github.com/lhl/claudecycles-revisited I used Codex w/ 5.2 xhigh and a relatively simple AGENTS.md - I have some session-analysis as well. The original replication was 47 minutes, th…

As described in the readme of your repo (did you read it?) your agent found the Knuth paper located one directory level above its working directory. So, you didn't produce a replication in 47 minutes, it just took around 30 minutes for your agent to find that you had the answer in a PDF in a nearby directory.

I wonder how common of a problem this will be in the future. The experiment will fail due to improper setup, the human will at best glance over the logs and declare victory, and everyone just believes.

Re: Claude's Cycles [pdf]

#315

Earlier quoted context omitted.

That is a descriptive surface level reduction. Now do the work to define what that actually means for the intelligence.

Nobody else in the thread is making an argument that relies on the distinction. "Intelligence" is used most commonly to refer to a class or collection of cognitive abilities. I don't think there is a consensus on an exact collection or specific class that the word covers, even if you consider specific scientific domains. LLMs have honestly been a fun way to explore that. They obviously have a "kind" of intelligence,…

The problem I see regarding LLMs is they are the extreme edge of what humans have created. They are trained on the outputs of intelligence and thought and its representation in language is this like parallel stream to intelligence that has pointers back to the underlying machine and semantics. The fact that LLMs are able to take that output and reverse engineer something that mimics the underlying machine that created that output is fascinating. But you can still see this machinery for what it is.

LLMs falls apart on really simple reasoning tasks because when there is no statistical mapping to a problem in its network it has to generate a massive amount of tokens to maybe find the right statistical match to this new concept. It is so slow. It is not something you or I would recognize as a process of logical reasoning. It is more like statistically brute forcing reason by way of its statistical echo.

So, I guess pattern recall is the right words. Or statistical pattern matching. Recall works if you view a trained model as memories, which is how I often model what they store in my own mind. So, it is... something. Maybe intelligence. Maybe just a really convincing simulation of the outputs of intelligence. Is there a difference? Fundamentally I think so.

Re: Claude's Cycles [pdf]

#316
post #277

Earlier quoted context omitted.

A very good point. For anyone not familiar with anterograde amnesia, the classical case is patient H.M. ( https://en.wikipedia.org/wiki/Henry_Molaison ), whose condition was researched by Brenda Milner.

> Near the end of his life, Molaison regularly filled in crossword puzzles.[16] He was able to fill in answers to clues that referred to pre-1953 knowledge. As for post-1953 information, he was able to modify old memories with new informations. For instance, he could add a memory about Jonas Salk by modifying his memory of polio.[2] That's fascinating!

The nature of memory is so cool, the idea that there are completely different systems governing the creation of wholesale "new" memories and the modification of existing concepts is fascinating to me because those things really do "feel" different in a qualitative sense, but having evidence that you're physically doing something different in those cases is really cool.

Re: Claude's Cycles [pdf]

#318

Earlier quoted context omitted.

On Google, just clicking "AI Mode" gives you a substantially smarter model, and it's still pretty weak. But I assume the OP wasn't talking about Google because it doesn't seem to make this mistake even in a search.

It was bing as that is the default for Edge as supplied on my work laptop. It doesn't do this now, but it does do something else quite weird: search: was val kilmer pregnant or in heat answer: Not pregnant Val Kilmer was not pregnant or in heat during the events of "Heat." His character, Chris Shiherlis, is involved in a shootout and is shot, which indicates he is not in a reproductive or mating state at that time. A…

Microsoft is bad at AI and this is a great example. I'm wondering if someone saw your post on HN and tried to hardcode a rule here, because I agree, it's nonsense. None of the actual AI companies are emitting nonsense like this.

Re: Claude's Cycles [pdf]

#319
post #253

Earlier quoted context omitted.

That's true but the capability to go back to an older iteration, reflect and find the correct solution (for odd numbers) is, in my book, a sign of undeniable intelligence.

Or, the ability to construct additional sentences influenced by prior ones.

Those additional sentences are fairly non-trivial to construct, would you agree?
Post reply on HN