The link to assistent ui in the article 404's. It should be https://github.com/assistant-ui/assistant-ui
My bad, I typed `-ai` instead of `-ui`. Its fixed now.
I want everything local – Building my offline AI workspace
181–190 of 294 posts
Re: I want everything local – Building my offline AI workspace
#182Earlier quoted context omitted.
I mean, why aren't the apps on your phone all just webapps, right? (Also, eww)
Mostly because native apps can track you far more invasively than web apps can, and companies are hungry for your private data.
Not sure I agree with that.
It's a lot better on battery life and superior experience, especially if you are traveling or around areas with bad cell service.
Cookies track me around on websites all the time + modern telemetry is pretty crazy.
Re: I want everything local – Building my offline AI workspace
#183Earlier quoted context omitted.
Also you get a sort of complete privacy that the data never leaves your home too whereas at best you would have to trust the AI cloud providers that they are not training or storing that data. Its just more freedom and privacy in that matter.
> whereas at best you would have to trust the AI cloud providers that they are not training or storing that data. Yeah, about that. They even illegally torrented entire databases, hide their crawlers. Crawl entire newspaper archives without permission. They didn't respect the rights of big media companies. But they're going to respect the little guy's of course because it says to in the T&Cs. Uh-huh. Also, openai alr…
I agree but I was just (repeating?) some argument that I heard that if the companies would actually not follow on their premise that they are actually safe if they said so (think amazon bedrock tos policy which says such)
Then it will cause an insane backlash and nobody would use the product. So it is in their interest to not train/record.
But yes I also agree with you. They are already torrenting :/ So pretty sure if they can do illegal stuff scott free, they might do this too idk,
And yeah this was why I was actually saying that local matters more tbh. You just get rid of such headache.
Re: I want everything local – Building my offline AI workspace
#184This is fantastic work. The focus on a local, sandboxed execution layer is a huge piece of the puzzle for a private AI workspace. The `coderunner` tool looks incredibly useful. A complementary challenge is the knowledge layer: making the AI aware of your personal data (emails, notes, files) via RAG. As soon as you try this on a large scale, storage becomes a massive bottleneck. A vector database for years of emails c…
That can't be the correct paper... I think you meant this: https://arxiv.org/abs/2506.08276
Yeah that's it. My bad lol
Re: I want everything local – Building my offline AI workspace
#185Any way to install this via just a container? Similar to a `docker compose up -d` that a lot of projects offer. Just download the docker-compose.yml file into a folder, run the command, and you're running. If you want to delete everything, just `docker compose down` and delete the folder, and the container and everything is gone. Anything similar to that? I don't want to run a random install.sh on my machine that doe…
There are similar commands for coderunner (not the UI frontend):
container image pull instavm/coderunner
container run --name coderunner --detach instavm/coderunner
Frontend (coderunner-ui) is not inside a docker as of now.
Re: I want everything local – Building my offline AI workspace
#186This is fantastic work. The focus on a local, sandboxed execution layer is a huge piece of the puzzle for a private AI workspace. The `coderunner` tool looks incredibly useful. A complementary challenge is the knowledge layer: making the AI aware of your personal data (emails, notes, files) via RAG. As soon as you try this on a large scale, storage becomes a massive bottleneck. A vector database for years of emails c…
This looks incredibly useful for making large-scale local AI truly practical.
Re: I want everything local – Building my offline AI workspace
#187Earlier quoted context omitted.
I think the local LLM scene is very fun and I enjoy following what people do. However every time I run local models on my MacBook Pro with a ton of RAM, I’m reminded of the gap between local hosted models and the frontier models that I can get for $20/month or nominal price per token from different providers. The difference in speed and quality is massive. The current local models are very impressive, but they’re sti…
I agree and disagree. Many of the best models are open source, just too big to run for most people. And there are plenty of ways to fit these models! A Mac Studio M3 Ultra with 512 GB unified memory though has huge capacity, and a decent chunk of bandwidth (800GB/s. Compare vs a 5090's ~1800GB/s). $10k is a lot of money, but that ability to fit these very large models & get quality results is very impressive. Perform…
You'll want to look at benchmarks rather than the theoretical maximum bandwidth available to the system. Apple has been using bandwidth as a marketing point but you're not always able to use that bandwidth amount depending on your workload. For example, the M1 Max has 400GB/s advertised bandwidth but the CPU and GPU combined cannot utilize all of it [1]. This means Strix Halo could actually be better for LLM inference than Apple Silicon if it achieves better bandwidth utilization.
[1] https://web.archive.org/web/20250516041637/https://www.anand...
Re: I want everything local – Building my offline AI workspace
#188Re: I want everything local – Building my offline AI workspace
#189This is fantastic work. The focus on a local, sandboxed execution layer is a huge piece of the puzzle for a private AI workspace. The `coderunner` tool looks incredibly useful. A complementary challenge is the knowledge layer: making the AI aware of your personal data (emails, notes, files) via RAG. As soon as you try this on a large scale, storage becomes a massive bottleneck. A vector database for years of emails c…
I have 26tb hardrives, 50gb doesnt scare me. Or should I be?
I think you'd want things in RAM for performance reasons but would love to be corrected by people with more knowledge/experience on the subject
Re: I want everything local – Building my offline AI workspace
#190Earlier quoted context omitted.
Mostly because native apps can track you far more invasively than web apps can, and companies are hungry for your private data.
Not sure I agree with that. It's a lot better on battery life and superior experience, especially if you are traveling or around areas with bad cell service. Cookies track me around on websites all the time + modern telemetry is pretty crazy.
The permissions that apps can get on Android even by default are pretty invasive, like querying other apps/processes and etc iirc...