Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenClaw’s memory is unreliable, and you don’t know when it will break

blog.nishantsoni.com

Post reply on HN
Print view View original

131–140 of 196 posts

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#131

Post by loehnsberg »

ambewas wrote: post #124
loehnsberg wrote: post #122

As long as there's no solution to the long-term memory problem, we will have a "country of geniuses in a data center" that are all suffering from anterograde amnesia (movie: Memento), which requires human hand-holding. I have experimented with a lot of hacks, like hierarchies of indexed md files, semantic DBs, embeddings, dynamic context retrieval, but none of this is really a comprehensive solution to get something…

You're right to be skeptical. Without a way to actually implement how the human brain processes experiences into a consolidated memory, we won't be able to solve the long term memory problem at all. Not with the current technology. An LLM context is a pretty well extended short term memory, and the trained network is a very nice comprehensive long term memory, but due to the way we currently train these networks, an…

I think if we want to build on what we have, instead of compaction at the end of the context window, the LLM would have to 'sleep', i.e. adjust its weights, then wake up with the last bits of the old context window in the new one, and have a 'feel' for what it did before through the change in weights. I just sense it's not that simple to get there, because simply updating the weights based on a single context sample risks degrading the weights of the whole network.

I like the idea of using small local model (or several) for tackling this problem, like low rank adaptation, but with current tech, I still have to piece this together or the small local models will forget old memories.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#132

Post by sonink »

loehnsberg wrote: post #122

As long as there's no solution to the long-term memory problem, we will have a "country of geniuses in a data center" that are all suffering from anterograde amnesia (movie: Memento), which requires human hand-holding. I have experimented with a lot of hacks, like hierarchies of indexed md files, semantic DBs, embeddings, dynamic context retrieval, but none of this is really a comprehensive solution to get something…

[dead]

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#133

Post by thepasch »

raincole wrote: post #98
thepasch wrote: post #10

It would’ve happened eventually anyway, but OpenClaw is basically what kickstarted the beginning of the end of token subsidies. It’s a almost begging to be used wastefully. And agents would miss and lose nothing without it. It’s devoid of a reason to exist.

I don't think this is fair (and I say this as someone who doesn't see much a point of OpenClaw). To me it's very obvious that Claude Code itself is the beginning of the end of token subsidies. When Claude Code was released, there was a community leaderboard where people competed who could waste the most tokens. Let that sinks. I know people, especially people who write code, like to blame "the other clueless people"…

Oh, of course we're all burning tokens like crazy. It's just that some are actually producing something of value with them, and some... are running OpenClaw.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#134

Post by fernandotakai »

chris_money202 wrote: post #97
jwpapi wrote: post #21

From my perspective there are some people that have never built real processes in their life that enjoy having some processes now. But agent processes are less reliable slower and less maintenable then a process that is well-defined and architectured and uses llm’s only where no other solution is sufficient. Classification, drafting, summarizing. I’ve had a Whatsapp assistant since 2023, jailbraked as easy assistant.…

This. So many junior engineers showing me AI flows that could just be a script with a few parameter inputs

yeah but unfortunately an AI flow can bring promotions, while scripts won't

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#135

Post by thepasch »

BeetleB wrote: post #44

If you look at my comment history, you'll see what seems to be someone defending OpenClaw (even though I stopped using it). I have some issues with the article, but I agree with some of the conclusions: It's great tinkering with it if you have time to spare, but not worth using weeks of your time trying to get a perfect setup. It's just not that reliable to use up so much of your time. I will say, it's still amongst…

Basically all modern models can do that, in _any_ conversation. Write a skill that teaches it how to use your API and where the endpoints are. Local models will need a harness (look up pi), but any frontier model can do this out of the box.

Writing a script to make a POST request is something assistants have been able to do for quite a while now.

And if you have a Claude subscription, you can use Dispatch to directly write to your PC's drive, no API needed.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#136

Post by drewstiff »

fg137 wrote: post #93
aunty_helen wrote: post #36

Earlier quoted context omitted.

The difference is I would have to do that myself. It has access to gdrive and cc and does it for me when I send it a message in chat. Sometimes when I’m out I even just send it voicys.

Method 1: run Claude Code in YOLO mode and use natural language instructions to get something done Method 2: send natural language instructions to OpenClaw to use Claude Code to do the same thing Sorry my tiny brain says to me method 2 is doing the same thing with extra steps

I can have multiple conversations on multiple topics always accessible via different discord channels, all with a shared memory, without that memory being held in a continually degrading context window.

One channel - reminders for medications, and recording my dosage. Another - "research this fancy new tech thing for me". Another - "let's continue work on that side project we started last week". And then in another - "create a dashboard of my meds dosage using that fancy new tech thing we were talking about yesterday". And of course finally "any urgent emails this morning?".

All without finding, creating, or setting up multiple apps or scripts for each individual task. If I have another idea, I just tell it what I want it to do, or ask it how we can make it happen.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#137

Post by SeriousM »

loehnsberg wrote: post #131
ambewas wrote: post #124

Earlier quoted context omitted.

You're right to be skeptical. Without a way to actually implement how the human brain processes experiences into a consolidated memory, we won't be able to solve the long term memory problem at all. Not with the current technology. An LLM context is a pretty well extended short term memory, and the trained network is a very nice comprehensive long term memory, but due to the way we currently train these networks, an…

I think if we want to build on what we have, instead of compaction at the end of the context window, the LLM would have to 'sleep', i.e. adjust its weights, then wake up with the last bits of the old context window in the new one, and have a 'feel' for what it did before through the change in weights. I just sense it's not that simple to get there, because simply updating the weights based on a single context sample…

Couldn't fitting solve the problem? That's what companies do: take a model as a base and train it on the specific data long enough so that it prefers the new data. Overfitting may be a thing but for personal use, I may want to have it work as I expected, every time.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#138

Post by Gareth321 »

loehnsberg wrote: post #122

As long as there's no solution to the long-term memory problem, we will have a "country of geniuses in a data center" that are all suffering from anterograde amnesia (movie: Memento), which requires human hand-holding. I have experimented with a lot of hacks, like hierarchies of indexed md files, semantic DBs, embeddings, dynamic context retrieval, but none of this is really a comprehensive solution to get something…

I agree. A key to human intelligence is our ability to adjust our weights in real-time. All knowledge becomes parametric knowledge - the knowledge stored inside the model. RAG is a messy workaround which requires making assumptions about what is needed to load from external sources before it is clear what is needed. Agentic loops can go some way to overcome this, but they are resource intensive, slow, prone to mistakes and deviations, and far less accurate. The secret sauce of an LLM is the vectorised weights. RAG is like putting a 1990s Honda Civic engine into a Ferrari. You can do it, but the result is quite terrible.

I think we will eventually end up with models which can be individually trained and customised on regular schedules. After that, real-time.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#139

Post by Gareth321 »

gbro3n wrote: post #130
loehnsberg wrote: post #122

As long as there's no solution to the long-term memory problem, we will have a "country of geniuses in a data center" that are all suffering from anterograde amnesia (movie: Memento), which requires human hand-holding. I have experimented with a lot of hacks, like hierarchies of indexed md files, semantic DBs, embeddings, dynamic context retrieval, but none of this is really a comprehensive solution to get something…

I've used open claw (just for learning, I agree with the author it's not reliable enough to do anything useful) but also have a similar daily summary routine which is a basic gemini api call to a personal mcp server that has access to my email, calendar etc. The latter is so much more reliable. Open claw flows sometimes nail it, and then the next day fails miserably. It seems like we need a way to 'bank' the correct…

That unreliability was why I gave up on OpenClaw. I tried hard to give it very simple tasks but it had a high degree of failure. Heartbeats and RAG are lightyears away from where they need to be. I'm not sure if this can be overcome using an application layer right now, but I trust that many people are trying, and I'm eager to see what emerges in the next year. In the mean time I know that they're working very hard on continuous learning - real-time updates to weights and parametric knowledge. It could be that in a year or so, we can all have customised models.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#140

Post by chris_money202 »

alfiedotwtf wrote: post #113
chris_money202 wrote: post #97

Earlier quoted context omitted.

This. So many junior engineers showing me AI flows that could just be a script with a few parameter inputs

Same thing could be said with SKILL.md yet they are highly useful... Yes you can automate via scripting, but interacting with a process using natural language because every instance could be different and not solid enough to write a spec for, is really handy. tl;dr: there's a place for "be liberal in what you receive and conservative in what you send", but only now have LLMs provided us with a viable way to make room…

I understand but there still is usually 80-95% of the skill flow that you can script out that is repeated. Script it out and simplify your skill, make it more stable, and provide more opportunity to scale it up or down i.e use stronger or weaker models if need be. We should be scripting and forming process first then seeing where we can put AI after that.
Post reply on HN