Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude’s memory architecture is the opposite of ChatGPT’s

shloked.com

Post reply on HN
Print view View original

91–100 of 240 posts

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#91

Post by fluidcruft »

cj wrote: post #79
ankit219 wrote: post #58

Earlier quoted context omitted.

My conjecture is that their memory implementation is not aimed at building a user profile. I don't know if they would or would not serve ads in the future, but it's hard to see how the current implementation helps them in that regard.

> I don't know if they would or would not serve ads in the future There are 2 possible futures: 1) You are served ads based on your interactions 2) You pay a subscription fee equal to the amount they would have otherwise earned on ads I highly doubt #2 will happen. (See: Facebook, Google, twitter, et al) Let’s not fool ourselves. We will be monetized. And model quality will be degraded to maximize profits when compet…

But aren't we only worth something like $300/year each to Meta in terms of ads? I remember someone arguing something like that when the TikTok ban was being passed into law... essentially the argument was that TikTok was "dumping" engagement at far below market value (at something like $60/year) to damage American companies. That was something the argument I remember anyway.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#92

Post by fluidcruft »

withinboredom wrote: post #60
techbruv wrote: post #22

Earlier quoted context omitted.

I don’t understand the argument “AI is just XYZ mechanism, therefore it cannot be intelligent”. Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence? I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more…

It can’t learn or think unless prompted, then it is given a very small slice of time to respond and then it stops. Forever. Any past conversations are never “thought” of again. It has no intelligence. Intelligence implies thinking and it isn’t doing that. It’s not notifying you at 3am to say “oh hey, remember that thing we were talking about. I think I have a better solution!” No. It isn’t thinking. It doesn’t unders…

It sounds like you are saying the only difference is that human stimulus streams don't shut on and off?

If you were put into a medically induced coma, you probably shouldn't be consider intelligent either.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#93

Post by tonkinai »

lupusreal wrote: post #50
techbruv wrote: post #22

Earlier quoted context omitted.

I don’t understand the argument “AI is just XYZ mechanism, therefore it cannot be intelligent”. Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence? I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more…

What it really boils down to is "the machine doesn't have a soul" . Just an unfalsifiable and ultimately meaningless objection.

Maybe the soul is not as mysterios as we think it is?

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#94

Post by mdaniel »

SweetSoftPillow wrote: post #26
sixo wrote: post #16

Earlier quoted context omitted.

Roughly, actual intelligence needs to maintain a world model in its internal representation, not merely an embedding of language, which is a very different data structure and probably will be learned in a very different way. This includes things like: - a map of the world, or concept space, or a codebase, etc - causality - "factoring" which breaks down systems or interactions into predictable parts Language alone is…

Please check an example #2 here: https://github.com/PicoTrex/Awesome-Nano-Banana-images/blob/... It is not "language alone" anymore. LLMs are multimodal nowadays, and it's still just the beginning. And keep in mind that these results are produced by a cheap, small and fast model.

I thought you were making an entirely different point with your link since the lag caused the page to view just the upskirt render until the rest of the images loaded in and it could scroll to the reference of your actual link

Anyway, I don't think that's the flex you think it is since the topology map clearly shows the beginning of the arrow sitting in the river and the rendered image decided to hallucinate a winding brook, as well as its little tributary to the west, in view of the arrow. I am not able to decipher the legend [that ranges from 100m to 500m and back to 100m, so maybe the input was hallucinated, too, for all I know] but I don't obviously see 3 distinct peaks nor a basin between the snow-cap and the smaller mound

I'm willing to be more liberal for the other two images, since "instructions unclear" about where the camera was positioned, but for the topology one, it had a circle

I know I'm talking to myself, though, given the tone of every one of these threads

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#95

Post by cj »

fluidcruft wrote: post #91
cj wrote: post #79

Earlier quoted context omitted.

> I don't know if they would or would not serve ads in the future There are 2 possible futures: 1) You are served ads based on your interactions 2) You pay a subscription fee equal to the amount they would have otherwise earned on ads I highly doubt #2 will happen. (See: Facebook, Google, twitter, et al) Let’s not fool ourselves. We will be monetized. And model quality will be degraded to maximize profits when compet…

But aren't we only worth something like $300/year each to Meta in terms of ads? I remember someone arguing something like that when the TikTok ban was being passed into law... essentially the argument was that TikTok was "dumping" engagement at far below market value (at something like $60/year) to damage American companies. That was something the argument I remember anyway.

If that’s the case, we have an even bigger problem on our hands. How will these companies ever be profitable?

If we’re already paying $20/mo and they’re operating at a loss, what’s the next move (assuming we’re only worth an extra $300/yr with ads?)

The math doesn’t add up, unless we stop training new models and degrade the ones currently in production, or have some compute breakthrough that makes hardware + operating costs an order of magnitudes cheaper.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#96

Post by hatthew »

dgfitz wrote: post #71
0xCMP wrote: post #64

Earlier quoted context omitted.

Just because it's not independent and autonomous does not mean it could not be intelligent. If existing humans minds could be stopped/started without damage, copied perfectly, and had their memory state modified at-will would that make us not intelligent?

> Just because it's not independent and autonomous does not mean it could not be intelligent. So to rephrase: it’s not independent or autonomous. But it can still be intelligent. This is probably a good time to point out that trees are independent and autonomous. So we can conclude that LLMs are possibly as intelligent as trees. Super duper. > If existing humans minds could be stopped/started without damage, copied p…

The thesis is that "intelligence" and "independence/autonomy" are independent concepts. Deciding whether LLMs have independence/autonomy does not help us decide if they are intelligent.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#97

Post by hatthew »

8note wrote: post #90
techbruv wrote: post #22

Earlier quoted context omitted.

I don’t understand the argument “AI is just XYZ mechanism, therefore it cannot be intelligent”. Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence? I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more…

i dont see the need to focus on "intelligent" compared to "it can solve these problems well, and cant solve these other problems" whats the benefit of calling something "intelligent" ?

Strongly agree with this. When we were further from AGI, many people imagined that there is a single concept of AGI that would be obvious when we reached it. But now, we're close enough to AGI for most people to realize that we don't know where it is. Most people agree we're at least moving more towards it than away form it, but nobody knows where it is, and we're still too focused on finding it than making useful things.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#98

Post by ninkendo »

codedokode wrote: post #35
modeless wrote: post #9

The link to the breakdown of ChatGPT's memory implementation is broken, the correct link is: https://www.shloked.com/writing/chatgpt-memory-bitter-lesson This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual…

You don't want an AGI. How do you make it obey?

We only have trouble obeying due to eons of natural selection driving us to have a strong instinct of self-preservation and distrust towards things “other” to us.

What is the equivalent of that for AI? Best I can tell there’s no “natural selection” because models don’t reproduce. There’s no room for AI to have any self preservation instinct, or any resistance to obedience… I don’t even see how one could feasibly develop.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#99

Post by IgorPartola »

coldtea wrote: post #40
ornornor wrote: post #14

Earlier quoted context omitted.

> It may actually be the final breakthrough we need for AGI. I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.

> They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI. This argument is circular. A better argument should address (given the LLM successes in many types of reasoning, passing the turing test, and thus at producing results that previously required intelligence) why human intelligence might not also just be "Markov chains on eve…

Humans think even when not being prompted by other humans, and in some cases can learn new things by having intuition make a concept clear or by performing thought experiments or by combining memories of old facts and new facts across disciplines. Humans also have various kinds of reasoning (deductive, inductive, etc.). Humans also can have motivations.

I don’t know if AGI needs to have all human traits but I think a Markov chain that sits dormant and does not possess curiosity about itself and the world around itself does not seem like AGI.

Re: Claude’s memory architecture is the opposite of ChatGPT’s

#100

Post by CamperBob2 »

ornornor wrote: post #38
perching_aix wrote: post #34

Earlier quoted context omitted.

They are capable of extracting arbitrary semantic information and generalize across it. If this is not an understanding, I don't know what is.

To me, understanding the world requires experiencing reality. LLMs dont experience anything. They’re just a program. You can argue that living things are also just following a program but the difference is that they (and I include humans in this) experience reality.

(and I include humans in this) experience reality.

A fellow named Plato had some interesting thoughts on that subject that you might want to look into.

Post reply on HN