Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude’s memory architecture is the opposite of ChatGPT’s

shloked.com

Post reply on HN
Print view View original

21–30 of 240 posts

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#21

Post by ForHackernews »

SweetSoftPillow wrote: post #15
ornornor wrote: post #14

Earlier quoted context omitted.

> It may actually be the final breakthrough we need for AGI. I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.

What is "actual intelligence" and how are you different from a Markov chain?

For one thing, I have internal state that continues to exist when I'm not responding to text input; I have some (limited) access to my own internal state and can reason about it (metacognition). So far, LLMs do not, and even when they claim they are, they are hallucinating https://transformer-circuits.pub/2025/attribution-graphs/bio...

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#22

Post by techbruv »

ornornor wrote: post #14
modeless wrote: post #9

The link to the breakdown of ChatGPT's memory implementation is broken, the correct link is: https://www.shloked.com/writing/chatgpt-memory-bitter-lesson This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual…

> It may actually be the final breakthrough we need for AGI. I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.

I don’t understand the argument “AI is just XYZ mechanism, therefore it cannot be intelligent”.

Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence?

I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more complex math problems than I can seems like a stretch.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#23

Post by aleph_minus_one »

WJW wrote: post #7
richwater wrote: post #2

ChatGPT is quickly approaching (perhaps bypassing?) the same concerns that parents, teachers, psychologists had with traditional social media. It's only going to get worse, but trying to stop the technological process will never work. I'm not sure what the answer is. That they're clearly optimizing for people's attention is more worrisome.

Seems like either a huge evolutionary advantage for the people who can exploit the (sometimes hallucinating sometimes not) knowledge machine, or else a huge advantage for the people who are predisposed to avoid the attention sucking knowledge machine. The ecosystem shifted, adapt or be outcompeted.

> Seems like either a huge evolutionary advantage for the people who can exploit the (sometimes hallucinating sometimes not) knowledge machine, or else a huge advantage for the people who are predisposed to avoid the attention sucking knowledge machine. The ecosystem shifted, adapt or be outcompeted.

Rather: use your time to learn serious, deep knowledge instead of wasting your time reading (and particularly: spreading) the science-fiction stories the AI bros tell all the time. These AI bros are insanely biased since they will likely loose a lot of money if these stories turn out to be false, or likely even if people stop believing in these science-fiction fairy tales.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#24

Post by sindercal »

ornornor wrote: post #18
SweetSoftPillow wrote: post #15

Earlier quoted context omitted.

What is "actual intelligence" and how are you different from a Markov chain?

What I mean is that the current generation of LLMs don’t understand how concepts relate to one another. Which is why they’re so bad at maths for instance. Markov chains can’t deduce anything logically. I can.

You and Chomsky are probably the last 2 persons on earth to believe that.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#25

Post by oasisaimlessly »

ornornor wrote: post #18
SweetSoftPillow wrote: post #15

Earlier quoted context omitted.

What is "actual intelligence" and how are you different from a Markov chain?

What I mean is that the current generation of LLMs don’t understand how concepts relate to one another. Which is why they’re so bad at maths for instance. Markov chains can’t deduce anything logically. I can.

The definition of 'Markov chain' is very wide. If you adhere to a materialist worldview, you are a Markov chain. [Or maybe the universe viewed as a whole is a Markov chain.]

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#26

Post by SweetSoftPillow »

sixo wrote: post #16
SweetSoftPillow wrote: post #15

Earlier quoted context omitted.

What is "actual intelligence" and how are you different from a Markov chain?

Roughly, actual intelligence needs to maintain a world model in its internal representation, not merely an embedding of language, which is a very different data structure and probably will be learned in a very different way. This includes things like: - a map of the world, or concept space, or a codebase, etc - causality - "factoring" which breaks down systems or interactions into predictable parts Language alone is…

Please check an example #2 here: https://github.com/PicoTrex/Awesome-Nano-Banana-images/blob/...

It is not "language alone" anymore. LLMs are multimodal nowadays, and it's still just the beginning.

And keep in mind that these results are produced by a cheap, small and fast model.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#29

Post by pityJuke »

qgin wrote: post #4

I love Claude's memory implementation, but I turned memory off in ChatGPT. I use ChatGPT for too many disparate things and it was weird when it was making associations across things that aren't actually associated in my life.

Exactly. The control over when to actually retrieve historical chats is so worthwhile. With ChatGPT, there is some slop from conversations I might have no desire to ever refer to again.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#30

Post by lyime »

ornornor wrote: post #14
modeless wrote: post #9

The link to the breakdown of ChatGPT's memory implementation is broken, the correct link is: https://www.shloked.com/writing/chatgpt-memory-bitter-lesson This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual…

> It may actually be the final breakthrough we need for AGI. I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.

How do you define "LLMs don't understand concepts"?

How do you define "understanding a concept" - what do you get if a system can "understand" concept vs not "understanding" a concept?

Post reply on HN