You can get good models that run fine on M1 32GB laptops just using Ollama App.
Or if you want numerous features on top of your local LLMS then Open WebUI would be my choice.
I want everything local – Building my offline AI workspace
171–180 of 294 posts
Re: I want everything local – Building my offline AI workspace
#172What is the Apple hardware being used here? I see Apple Silicon but not the configuration.. what did I miss
Re: I want everything local – Building my offline AI workspace
#173Earlier quoted context omitted.
> A vector database for years of emails can easily exceed 50GB. In 2025 I would consider this a relatively meager requirement.
Yeah, that's a fair point at first glance. 50GB might not sound like a huge burden for a modern SSD. However, the 50GB figure was just a starting point for emails. A true "local Jarvis," would need to index everything: all your code repositories, documents, notes, and chat histories. That raw data can easily be hundreds of gigabytes. For a 200GB text corpus, a traditional vector index can swell to >500GB. At that poi…
While your aims are undoutably sincere, in practice for the 'local ai' target people building their own rigs usually have. 4TB or more fast ssd storage.
The bottom tier (not meant disparagingly) are people running diffusion models as these do not have the high vram requirements. They generate tons of images or video, going form a one-click instally like Easydiffusion to very sophisticated workflows in comfyui.
For those going the LLM route, which would be your target audience, they quickly run into the problemm that to go beyond toying around, the hardware and software requirements and expertise grows exponential beyong just toying around with small, highly quantized model with small context windows.
Inlight of the typical enthusiast investments in this space, the few TB of fast storage will pale in comparison to the rest of the expenses.
Again, your work is absolutely valuable, it is just that the storage space requirement for the vector store in this particular scenario is not your strongest card to play.
Re: I want everything local – Building my offline AI workspace
#174Earlier quoted context omitted.
I'm a desktop guy, considering the switch to a laptop-only setup, what would I miss ?
For $10k, you too can get the power of a $2k desktop, and enjoy burning your lap everyday, or something like that. If I were to do local compute and wanted to use my laptop, I would only consider a setup where I ssh in to my desktop. So I guess only difference from saas llm would be privacy and the cool factor. And rate limits, and paying more if you go over, etc.
> For $10k, you too can get the power of a $2k desktop,
Even M1 MBP 32GB performance is pretty impressive for its age and you can get them for well I have one.
I use these models: gpt-oss, llama3.2, deepseek, granite3.3
They all work fine and speed is not an issue. The recent Ollama app means I can have document/image processing with the LLM as well.
Re: I want everything local – Building my offline AI workspace
#175Any way to install this via just a container? Similar to a `docker compose up -d` that a lot of projects offer. Just download the docker-compose.yml file into a folder, run the command, and you're running. If you want to delete everything, just `docker compose down` and delete the folder, and the container and everything is gone. Anything similar to that? I don't want to run a random install.sh on my machine that doe…
I believe a flatpack or appimage is what you're looking for.
Re: I want everything local – Building my offline AI workspace
#176Earlier quoted context omitted.
> anything you pick up second-hand will still deprecate at that pace Not really? The people who do local inference most (from what I've seen) are owners of Apple Silicon and Nvidia hardware. Apple Silicon has ~7 years of decent enough LLM support under it's belt, and Nvidia is only now starting to depreciate 11-year-old GPU hardware in drivers. If you bought a decently powerful inference machine 3 or 5 years ago, it'…
People on HN do a lot of wishful thinking when it comes to the macOS LLM situation. I feel like most of the people touting the Mac’s ability to run LLMs are either impressed that they run at all, are doing fairly simple tasks, or just have a toy model they like to mess around with and it doesn’t matter if it messes up. And that’s fine! But then people come into the conversation from Claude Code and think there’s a wa…
> I feel like most of the people touting the Mac’s ability to run LLMs are either impressed that they run at all, are doing fairly simple tasks, or just have a toy model they like to mess around with and it doesn’t matter if it messes up.
I feel like you haven't actually used it. Your comment may have been true 5 years ago.
> If you want an assistant you can talk to that will give you advice or help you with arbitrary tasks for work, that’s not something that’s on the menu.
You can use a RAG approach (eg. Milvus) and also LoRA templates to dramatically improve the accuracy of the answer if needed.
Locally you can run multiple models, multiple times without having to worry about costs.
You also have the likes of Open WebUI which builds numerous features on top of an interface if you don't want to do coding.
I have a very old M1 MBP 32GB and I have numerous applications built to do custom work. It does the job the fine and speed is not an issue. Not good enough to do a LoRA build but I have a more recent laptop for that.
I doubt I am the only one.
Re: I want everything local – Building my offline AI workspace
#177Re: I want everything local – Building my offline AI workspace
#178Earlier quoted context omitted.
Yeah, that's a fair point at first glance. 50GB might not sound like a huge burden for a modern SSD. However, the 50GB figure was just a starting point for emails. A true "local Jarvis," would need to index everything: all your code repositories, documents, notes, and chat histories. That raw data can easily be hundreds of gigabytes. For a 200GB text corpus, a traditional vector index can swell to >500GB. At that poi…
While your aims are undoutably sincere, in practice for the 'local ai' target people building their own rigs usually have. 4TB or more fast ssd storage. The bottom tier (not meant disparagingly) are people running diffusion models as these do not have the high vram requirements. They generate tons of images or video, going form a one-click instally like Easydiffusion to very sophisticated workflows in comfyui. For th…
Everyone benefits from focusing on efficiency and finding better ways of doing things. Those people with 4TB+ of fast storage can now do more than they could before as can the "bottom tier."
It's a breath of fresh air anytime someone finds a way to do more with less rather than just wait for things to get faster and cheaper.
Re: I want everything local – Building my offline AI workspace
#179you might want to check out what we built -> https://inference.sh supports most major open source/weight models from wan 2.2 video, qwen image, flux, most llms, hunyan 3d etc.. works in a containerized way locally by allowing you to bring your own gpu as an engine (fully free) or allows you to rent remote gpu/pool from a common cloud in case you want to run more complex models. for each model we tried to add quantize…
The website is very confusing. Where can I download the application? Is there a GitHub repository?
Re: I want everything local – Building my offline AI workspace
#180Earlier quoted context omitted.
While your aims are undoutably sincere, in practice for the 'local ai' target people building their own rigs usually have. 4TB or more fast ssd storage. The bottom tier (not meant disparagingly) are people running diffusion models as these do not have the high vram requirements. They generate tons of images or video, going form a one-click instally like Easydiffusion to very sophisticated workflows in comfyui. For th…
Everyone benefits from focusing on efficiency and finding better ways of doing things. Those people with 4TB+ of fast storage can now do more than they could before as can the "bottom tier." It's a breath of fresh air anytime someone finds a way to do more with less rather than just wait for things to get faster and cheaper.
Of course. And I am not arguing against that at all. Just like if someone makes an inference runtime that is 4% faster, I'll take that win. But would it be the decisive factor in my choice? Only if that was my bottleneck, my true constraint.
All I tried to convey was that for most of the people in the presented scenario (personal emails etc.) , a 50 or even 500GB storage requirement is not going to be that primary constraint. So the suggestion was the marketing for this usecase might be better spotlighting also something else.