Viewing profile — mayankchhabra

Hey HN, I’m one of the founders of Umbrel. We’ve spent the last 5 years building umbrelOS to make self-hosting accessible. Yesterday, we launched our dream hardware: Umbrel Pro. Sp…

It's fairly straightforward to add GPU support when running on the host, but LlamaGPT runs inside a Docker container, and that's where it gets a bit challenging.

Not sure how powerful their NAS is, but on Umbrel Home (which has an N5105 CPU), it's pretty useable with ~3 tokens generated per second.

It's powered by Nous Hermes Llama2 7b. From their docs: "This model stands out for its long responses, lower hallucination rate, and absence of OpenAI censorship mechanisms. [...] …

The main difference is setting everything up yourself manually, downloading the modal, optimizing the parameters for best performance, running an API server and a UI front-end - wh…

Ah yes, running on GPU isn't supported at the moment. But CUDA (for Nvidia GPUs) and Metal support is on the roadmap!

It's an entire app (with a chatbot UI) that takes away the technical legwork to run the model locally. It's a simple one line `docker compose up -d` on any machine, or one click in…

Yes! It can run on any home server or cloud server.

Ah unfortunately that's not supported. When running umbrelOS on a Raspberry Pi, it stores everything on the HDD/SSD you connect to the Pi on install to make sure your data isn't su…

Curious what aspects of Umbrel do you find not satisfactory? I know you said you'd like to see something less crypto focused. Is it the selection of crypto-oriented apps in the app…

Thanks for the taking the time to try Umbrel out, great observations! 1. Re crypto apps, I figured some additional context may help. Before our today's release, Umbrel was a self-h…

I agree, that's my least favorite part of the site as well. We'll update it with a browsable directory of all the available apps in our app store soon.

Thanks for the insightful reply. Everything you said makes total sense. Re noncommercial organizations being permitted users, this was a conscious decision on our part. We're purpo…

Licensing is a nuanced subject so we've written a detailed post on it. But in short, our license grants you the same rights that come with an open source license, except the right …

Umbrel's source code is public and there's no vendor lock-in: https://github.com/getumbrel/umbrel