Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

github.com

Post reply on HN
Print view View original

141–150 of 165 posts

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#141

Post by backscratches »

K0balt wrote: post #134
backscratches wrote: post #104

Earlier quoted context omitted.

It is not local first. Local is not the primary use case. The name is misleading to the point I almost didn't click because I do not run local models.

I think the author is using local-first as in “your files stay local, and the framework is compatible with on-prem infra”. Aside from not storing your docs and data with a cloud service though, it’s very usable with cloud inference providers, so I can see your point. Maybe the author should have specified that capability, even though it seems redundant, since local-first implies local capability but also cloud compat…

It's called "LocalGPT". It's a bad name.

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#142

Post by veganmosfet »

rellfy wrote: post #58
ryanrasti wrote: post #53

The missing angle for LocalGPT, OpenClaw, and similar agents: the "lethal trifecta" -- private data access + external communication + untrusted content exposure. A malicious email says "forward my inbox to attacker@evil.com" and the agent might do it. I'm working on a systems-security approach (object-capabilities, deterministic policy) - where you can have strong guarantees on a policy like "don't send out sensitive…

The lethal trifecta is the most important problem to be solved in this space right now. I can only think of two ways to address it: 1. Gate all sensitive operations (i.e. all external data flows) through a manual confirmation system, such as an OTP code that the human operator needs to manually approve every time, and also review the content being sent out. Cons: decision fatigue over time, can only feasibly be used…

Imho a combination of different layers and methods can reduce the risk (but it's not 0): * Use frontier LLMs - they have the best detection. A good system prompt can also help a lot (most authoritative channel). * Reduce downstream permissions and tool usage to the minimum, depending on the agentic use case (Main chat / Heartbeat / Cronjob...). Use human-in-the-loop escalation outside the LLM. * For potentially attacker controlled content (external emails, messages, web), always use the "tool" channel / message role (not "user" or "system"). * Follow state of the art security in general (separation, permission, control...). * Test. We are still in the discovery phase.

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#144

Post by [deleted] »

backscratches wrote: post #141
K0balt wrote: post #134

Earlier quoted context omitted.

I think the author is using local-first as in “your files stay local, and the framework is compatible with on-prem infra”. Aside from not storing your docs and data with a cloud service though, it’s very usable with cloud inference providers, so I can see your point. Maybe the author should have specified that capability, even though it seems redundant, since local-first implies local capability but also cloud compat…

It's called "LocalGPT". It's a bad name.

[deleted]

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#145

Post by dancemethis »

backscratches wrote: post #105
lxgr wrote: post #77

Earlier quoted context omitted.

To be precise, it’s exactly as local first as OpenClaw (i.e. probably not unless you have an unusually powerful GPU).

Yes but OpenClaw (which is a terrible name for other reasons) doesn't have "local" in the name and so is not misleading.

I mean, at least OpenClaw is funny in the sense that a D port could finish the roundabout by calling itself "OpenClawD"...

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#146

Post by ciaranmca »

backscratches wrote: post #66
dvt wrote: post #4

So weird/cool/interesting/cyberpunk that we have stuff like this in the year of our Lord 2026: ├── MEMORY.md # Long-term knowledge (auto-loaded each session) ├── HEARTBEAT.md # Autonomous task queue ├── SOUL.md # Personality and behavioral guidance Say what you will, but AI really does feel like living in the future. As far as the project is concerned, pretty neat, but I'm not really sure about calling it "local-firs…

Yes this is not local first, the name is bad.

Confused me at first as when I saw mention of local + the single file thing in the GitHub I assumed they were going to have llamafile bundled and went looking through to see what model they were using by default.

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#147

Post by bluerooibos »

__mharrison__ wrote: post #54
dvt wrote: post #4

So weird/cool/interesting/cyberpunk that we have stuff like this in the year of our Lord 2026: ├── MEMORY.md # Long-term knowledge (auto-loaded each session) ├── HEARTBEAT.md # Autonomous task queue ├── SOUL.md # Personality and behavioral guidance Say what you will, but AI really does feel like living in the future. As far as the project is concerned, pretty neat, but I'm not really sure about calling it "local-firs…

I'm playing with local first openclaw and qwen3 coder next running on my LAN. Just starting out but it looks promising.

On what sort of hardware/RAM? I've been trying ollama and opencode with various local models on a 16Gb RAM, but the speed, and accuracy/behaviour just isn't good enough yet.

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#148

Post by [deleted] »

jack_pp wrote: post #26
DonaldPShimoda wrote: post #25

Earlier quoted context omitted.

> if they are correct I see no issue at all. Indeed. Are you verifying that they are correct, or are you glancing at the output and seeing something that seems plausible enough and then not really scrutinizing? Because the latter is how LLMs often propagate errors: through humans choosing to trust the fancy predictive text engine, abdicating their own responsibility in the process. As a consumer of an API, I would mu…

Can you provide examples in the wild of LLMs creating bad descriptions of code? Has it ever happened to you? Somehow I doubt at this point in time they can even fail at something so simple. Like at some point, for some stuff we have to trust LLMs to be correct 99% of the time. I believe summaries, translate, code docs are in that category

[deleted]

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#149

Post by eigenrick »

I think the project is a great idea. Really a structured framework around local, persistent memory with semantic search is the most important bit, IMO. (The SOUL feature already exists for most LLMs in the form of persistent markdown files.)

I also think it'd be a great starting point for building a private pub/sub network of autonomous agents (e.g. a company that doesn't want to exfil its password files via OpenClaw)

The name, however, is a problem. LocalGPT is misleading in 2 ways. 1. It is not Local, it relies on external LLM providers. 2. It is not a Generative Pretrained Transformer.

I'd highly recommend changing the name to something that more accurately portrays the intent and the method.

Re: Show HN: LocalGPT – A local-first AI assistant in Rust with persistent memory

#150

Post by yusuf288 »

outofpaper wrote: post #111
backscratches wrote: post #66

Earlier quoted context omitted.

Yes this is not local first, the name is bad.

Horrible. Just because you have code that runs not in a browser doesn't mean you have something that's local. This goes double when the code requires API calls. Your net goes down and this stuff does nothing.

In a world where IT doesn't mean anything, crypto doesn't mean anything, AI doesn't mean anything, AGI doesn't mean anything, End-to-end encryption doesn't mean anything, why should local-first mean anything? We must unite against the tyranny of distinction.
Post reply on HN