Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new study of a bot running a store finds it is friendly but not very smart

nytimes.com

Post reply on HN
Print view View original

11–20 of 71 posts

Re: A new study of a bot running a store finds it is friendly but not very smart

#11

Post by SwellJoe »

criddell wrote: post #4
SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

> People just really don't understand that LLMs do not "remember" anything. Sure, but like you said, the LLM is just one part of the AI system. The occipital lobe of your brain helps out with vision but doesn't help with your ability to verbalize.

You're anthropomorphizing the LLMs, just like most writing from non-technical press, including this article, does.

LLMs do not function very much like human brains, and when you try to map the pieces of an inference system on to human cognitive systems, you're misleading yourself (and maybe others).

Re: A new study of a bot running a store finds it is friendly but not very smart

#12

Post by dom96 »

SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

They can have memory, it's just limited to their context window.

Re: A new study of a bot running a store finds it is friendly but not very smart

#13

Post by jpiasolutions »

SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

The memory framing is right, but the real gap isn't recall, it's the missing feedback loop tying actions to outcomes. Even with perfect retrieval these agents don't update from a decision that lost money last week; they re-derive from context every step, so the same mistake is always one dropped detail away. And having built retrieval-backed agents, that's where "bolt a database on the side" breaks: similarity search returns the closest chunk, rarely the contextually-right one. Deciding what to write to memory and when to surface it is the actual product, harder than the retrieval itself.

Re: A new study of a bot running a store finds it is friendly but not very smart

#14

Post by oliver236 »

SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

can someone explain how leapold aschenbrenner proposes a solution to this in situational awareness?

Re: A new study of a bot running a store finds it is friendly but not very smart

#15

Post by walrus01 »

SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

From the article:

> Luna has failed miserably in that mission and is down $62,000. Mr. Petersson and Mr. Backlund said they thought Luna would eventually get smarter and more business-savvy and were pleased its friendliness has held steady.

I am extremely skeptical that whatever LLM they're running this on has sufficient context window size to handle multiple months of all possible activities of running a retail business. Even if it's keeping extensive "notes" for its future self to read, it's going to be like running a store with constant amnesia.

Re: A new study of a bot running a store finds it is friendly but not very smart

#16

Post by criddell »

SwellJoe wrote: post #11
criddell wrote: post #4

Earlier quoted context omitted.

> People just really don't understand that LLMs do not "remember" anything. Sure, but like you said, the LLM is just one part of the AI system. The occipital lobe of your brain helps out with vision but doesn't help with your ability to verbalize.

You're anthropomorphizing the LLMs, just like most writing from non-technical press, including this article, does. LLMs do not function very much like human brains, and when you try to map the pieces of an inference system on to human cognitive systems, you're misleading yourself (and maybe others).

An AI as a system can have memory even if some sub-system does not.

Re: A new study of a bot running a store finds it is friendly but not very smart

#17

Post by Joel_Mckay »

SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

For LLM, keeping the model centered by constantly resetting its vector context helps reduce hallucinations by around 23%. It improved the chat dialogue users experienced, but also exposed fundamental limits within the models compaction.

Have a great day =3

Re: A new study of a bot running a store finds it is friendly but not very smart

#18

Post by Dylan16807 »

criddell wrote: post #4
SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

> People just really don't understand that LLMs do not "remember" anything. Sure, but like you said, the LLM is just one part of the AI system. The occipital lobe of your brain helps out with vision but doesn't help with your ability to verbalize.

And then they explained in detail why these supplementary systems fall below memory. Would you like to address that argument? If not your quick response to the first sentence is not very useful.

Re: A new study of a bot running a store finds it is friendly but not very smart

#20

Post by Joel_Mckay »

walrus01 wrote: post #15
SwellJoe wrote: post #3

People just really don't understand that LLMs do not "remember" anything. They have no memory. They can't have memory. The AI companies bolt a database on the side with instructions for the model to query the database for stuff in its "memory", but if it's not in its active context it doesn't know what to query for. It's not like a human where everything is kind of floating around and we mostly know what we know. The…

From the article: > Luna has failed miserably in that mission and is down $62,000. Mr. Petersson and Mr. Backlund said they thought Luna would eventually get smarter and more business-savvy and were pleased its friendliness has held steady. I am extremely skeptical that whatever LLM they're running this on has sufficient context window size to handle multiple months of all possible activities of running a retail busi…

90% of what people communicate is not verbal, and requires minimum empathy to understand the context.

This includes customers, staff, and community peers. Probably should shutter the entire division to mitigate future brand damage. =3

Post reply on HN