Live data from Hacker News

OpenClaw’s memory is unreliable, and you don’t know when it will break

blog.nishantsoni.com

171–180 of 196 posts

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#171
post #90

I actually quite enjoy the OpenClaw. Although the recent CC crackdown has caused me to try different LLM providers which aren't that reliable but anyways, here are few things I do with it, in all separate groups. * Telegram Health Group, created an agent to help me track sleep, recommend my supplements based on my location, remind me in the morning and evening to monitor my food. I send it images of what I eat and it…

[dead]

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#172
I’ve had some success with claude cli agents at some scale with a memory architecture - but it roughly reads like a massive index, where it crawls through a trail of breadcrumbs to piece together all the info it needs to do a task. It’s fairly tedious to maintain, and it’s always a battle maintaining reasonable context size and token spend.

I’d say it’s like 85% reliable on any given task, and since I supervise it, this is good enough for me. But for something to be useful autonomously, that number needs to be several 9’s to be useful at all, and we’re no world near that yet.

I’m currently watching someone trying and failing to roll openclaw out at scale in an org and they believe in it so much it’s very difficult to convince them even with glaring evidence staring them in the face that it will not work

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#173

Earlier quoted context omitted.

This is interesting. I haven't used OpenClaw but I set up my own autonomous agent using Codex + ChatGPT Plus + systemd + normal UNIX email and user account infrastructure. And it's been working great! I'm very happy with it. It's been doing all kinds of tasks for me, effectively as an employee of my company. I haven't seen any issues with memory so far. Using one long rolling context window, a diary and a markdown wi…

Any chance you might put this on GH? Sounds really interesting.

Maybe but it's so simple I'm not sure it's worth it. You can easily make your own!

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#174

I'm using openclaw as a personal development bot, which is pretty useful. It pings me throughout the day using crons to complete tasks and follows up on them. But aside from that, it is a very unreliable piece of software. I'm constantly having to fix it, or track down correct configurations. It can just decide to randomly edit it's own config, uses incorrect json keys and then the whole thing is dead. Or it blows th…

Amen. I love the concept but I've never hosted such a terrible piece of software. Every update breaks something new or introduces another "anti-feature" that's enabled by default. The documentation is often lagging behind and the changelog has such a low signal to noise ratio that you need a LLM to figure out what upgrading will break this time. For now I've just given up on updates and I've been patching bugs direct…

> If OpenClaw is the future of software I'm honestly a bit scared for the industry.

I think it's mainly the industry wannabes gathering around a "sexy" brand name again, when they're really more interested in "AI as personal assistants".

OpenClaw just has the most traction despite being a hot mess, because the people hyping it up don't know how bad the codebase is, or because they want to launch something first and switch it over to a more credible alternative after.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#175

My biggest issue with OpenClaw is everyone talks about doing things with it but doesn't explain what it actually is doing. First of all is not an LLM, you're beholden to an api or local llm limitations. Second of all it's always calendars, email replies, summarizing. You do not need an LLM for that, and an LLM doesn't make it easier either. It sounds like executive cosplay, not productivity. Everything I see people t…

I'm gonna die shaking my fists at clouds pretty soon, but amen. I had a guy in work Slack evangelizing about this in January when it first came out and he's talking about automatically ordering groceries and getting reminders to do laundry and shit and I just don't get it. Sure, that's a use case, but I figured out how to feed, clothe, and house myself without digital assistance 25 years ago. How'd you live this long…

> I had a guy in work Slack evangelizing about this in January when it first came out and he's talking about automatically ordering groceries and getting reminders to do laundry and shit and I just don't get it. Sure, that's a use case, but I figured out how to feed, clothe, and house myself without digital assistance 25 years ago. How'd you live this long if you can't eat without a machine telling you what to buy and how to cook?

Some people have developed personal coping strategies for neurodivergence, and can probably do better with some AI assistance.

I myself tend to live by my calendar. Even then, I may eventually forget to follow up on something until it's too late, because I'm busy or overwhelmed with other things.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#176

As long as there's no solution to the long-term memory problem, we will have a "country of geniuses in a data center" that are all suffering from anterograde amnesia (movie: Memento), which requires human hand-holding. I have experimented with a lot of hacks, like hierarchies of indexed md files, semantic DBs, embeddings, dynamic context retrieval, but none of this is really a comprehensive solution to get something…

You're right to be skeptical. Without a way to actually implement how the human brain processes experiences into a consolidated memory, we won't be able to solve the long term memory problem at all. Not with the current technology. An LLM context is a pretty well extended short term memory, and the trained network is a very nice comprehensive long term memory, but due to the way we currently train these networks, an…

How well do LoRAs work for this using something like Thinking Machine's Tinker?

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#177
post #38

Earlier quoted context omitted.

Amen. I love the concept but I've never hosted such a terrible piece of software. Every update breaks something new or introduces another "anti-feature" that's enabled by default. The documentation is often lagging behind and the changelog has such a low signal to noise ratio that you need a LLM to figure out what upgrading will break this time. For now I've just given up on updates and I've been patching bugs direct…

I know Twitter has been talking up Hermes Agent by Nous Research a lot or id recommend building your own agent off of Pi.

Thanks for this suggestion, I installed it yesterday after seeing this comment and this surely is a breath of fresh air! It appears that everything is designed reasonably well from the ground up. It’s more limited, but what’s there works well.

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#178

I'm using openclaw as a personal development bot, which is pretty useful. It pings me throughout the day using crons to complete tasks and follows up on them. But aside from that, it is a very unreliable piece of software. I'm constantly having to fix it, or track down correct configurations. It can just decide to randomly edit it's own config, uses incorrect json keys and then the whole thing is dead. Or it blows th…

Amen. I love the concept but I've never hosted such a terrible piece of software. Every update breaks something new or introduces another "anti-feature" that's enabled by default. The documentation is often lagging behind and the changelog has such a low signal to noise ratio that you need a LLM to figure out what upgrading will break this time. For now I've just given up on updates and I've been patching bugs direct…

Hermes Agent

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#179

Earlier quoted context omitted.

Same thing could be said with SKILL.md yet they are highly useful... Yes you can automate via scripting, but interacting with a process using natural language because every instance could be different and not solid enough to write a spec for, is really handy. tl;dr: there's a place for "be liberal in what you receive and conservative in what you send", but only now have LLMs provided us with a viable way to make room…

I understand but there still is usually 80-95% of the skill flow that you can script out that is repeated. Script it out and simplify your skill, make it more stable, and provide more opportunity to scale it up or down i.e use stronger or weaker models if need be. We should be scripting and forming process first then seeing where we can put AI after that.

That’s actually an awesome idea and totally helps to reduce wasting context size - move repeatable instructions to a SKILL.md, and once they’re repeatable and no longer have variability to input, turn it into a tool! Rinse repeat.

Oh nice, you could even eventually turn the whole process including inference into an app so that you’ve cut out the LLM from the whole process saving you execution time

Re: OpenClaw’s memory is unreliable, and you don’t know when it will break

#180

Earlier quoted context omitted.

Same thing could be said with SKILL.md yet they are highly useful... Yes you can automate via scripting, but interacting with a process using natural language because every instance could be different and not solid enough to write a spec for, is really handy. tl;dr: there's a place for "be liberal in what you receive and conservative in what you send", but only now have LLMs provided us with a viable way to make room…

I understand but there still is usually 80-95% of the skill flow that you can script out that is repeated. Script it out and simplify your skill, make it more stable, and provide more opportunity to scale it up or down i.e use stronger or weaker models if need be. We should be scripting and forming process first then seeing where we can put AI after that.

The AI for everything thinking is really easy to let infect you. I was trying to figure out how to make some SQL alerting easier to understand quickly. The first thing my brain went to was "oh just shove it into an LLM to pull out the info of what the query is doing". And it unfortunately wasn't until after I said that out loud that I realized that was a stupid idea when you could just run a SQL parser over the query and pull the table names out that way. Far faster, more cost effective, and reliable than asking an LLM to do it.
Post reply on HN