I want everything local – Building my offline AI workspace • Hacker News Board

I want everything local – Building my offline AI workspace

#1

Post by mkagenius » Fri, Aug 08, 2025, 6:19 PM UTC

I want everything local – Building my offline AI workspace
instavm.io

Re: I want everything local – Building my offline AI workspace

#2

Post by shaky » Fri, Aug 08, 2025, 6:39 PM UTC

This is something that I think about quite a bit and am grateful for this write-up. The amount of friction to get privacy today is astounding.

Re: I want everything local – Building my offline AI workspace

#3

Post by noelwelsh » Fri, Aug 08, 2025, 6:44 PM UTC

It's the hardware more than the software that is the limiting factor at the moment, no? Hardware to run a good LLM locally starts around $2000 (e.g. Strix Halo / AI Max 395) I think a few Strix Halo iterations will make it considerably easier.

Re: I want everything local – Building my offline AI workspace

#4

Post by ahmedbaracat » Fri, Aug 08, 2025, 6:45 PM UTC

Thanks for sharing. Note that the GitHub at the end of the article is not working…

Re: I want everything local – Building my offline AI workspace

#5

Post by colecut » Fri, Aug 08, 2025, 6:47 PM UTC

noelwelsh wrote: post #3

It's the hardware more than the software that is the limiting factor at the moment, no? Hardware to run a good LLM locally starts around $2000 (e.g. Strix Halo / AI Max 395) I think a few Strix Halo iterations will make it considerably easier.

This is rapidly improving

https://simonwillison.net/2025/Jul/29/space-invaders/

Re: I want everything local – Building my offline AI workspace

#6

Post by navbaker » Fri, Aug 08, 2025, 6:49 PM UTC

Open Web UI is a great alternative for a chat interface. You can point to an OpenAI API like vLLM or use the native Ollama integration and it has cool features like being able to say something like “generate code for an HTML and JavaScript pong game” and have it display the running code inline with the chat for testing

Re: I want everything local – Building my offline AI workspace

#7

Post by dmezzetti » Fri, Aug 08, 2025, 6:52 PM UTC

I built TxtAI with this philosophy in mind: https://github.com/neuml/txtai

Re: I want everything local – Building my offline AI workspace

#8

Post by ramesh31 » Fri, Aug 08, 2025, 6:53 PM UTC

noelwelsh wrote: post #3

It's the hardware more than the software that is the limiting factor at the moment, no? Hardware to run a good LLM locally starts around $2000 (e.g. Strix Halo / AI Max 395) I think a few Strix Halo iterations will make it considerably easier.

>Hardware to run a good LLM locally starts around $2000 (e.g. Strix Halo / AI Max 395) I think a few Strix Halo iterations will make it considerably easier.

And "good" is still questionable. The thing that makes this stuff useful is when it works instantly like magic. Once you find yourself fiddling around with subpar results at slower speeds, essentially all of the value is gone. Local models have come a long way but there is still nothing even close to Claude levels when it comes to coding. I just tried taking the latest Qwen and GLM models for a spin through OpenRouter with Cline recently and they feel roughly on par with Claude 3.0. Benchmarks are one thing, but reality is a completely different story.