My Homelab AI Dev Platform

Then, I said homelab AI, I thought it's an interesting post about local GPU setup (and I am really interested in this topic).. but no, just another hype post about how to use whatever-code...

Re: My Homelab AI Dev Platform

#5

Post by reactordev » Mon, Jun 15, 2026, 4:23 PM UTC

fazgha wrote: post #4

So first post in the blog, and it went directly HN frontpage. Then, I said homelab AI, I thought it's an interesting post about local GPU setup (and I am really interested in this topic).. but no, just another hype post about how to use whatever-code...

you can't explain the HN hug. You feel it, or your servers do.

Re: My Homelab AI Dev Platform

#6

Post by variety8675 » Mon, Jun 15, 2026, 4:24 PM UTC

How do you run inference for Open Code? What models are you running

Re: My Homelab AI Dev Platform

#7

Post by rsgm » Mon, Jun 15, 2026, 4:29 PM UTC

fazgha wrote: post #4

So first post in the blog, and it went directly HN frontpage. Then, I said homelab AI, I thought it's an interesting post about local GPU setup (and I am really interested in this topic).. but no, just another hype post about how to use whatever-code...

I looked into running local models last month. They just aren't quite there for agentic tool use workflows without spending a small fortune. I'm hopeful smaller local models get much better soon.

I was also hoping to put out another post on my homelab setup, it has some neat stuff, but I haven't had a chance to finish it.

Re: My Homelab AI Dev Platform

#8

Post by _def » Mon, Jun 15, 2026, 4:45 PM UTC

I wonder how gitops is done with docker compose

Re: My Homelab AI Dev Platform

#9

Post by dlxfoo » Mon, Jun 15, 2026, 4:47 PM UTC

Im doing something very similar. Running my OpenCode on a proxmox lxc. I have an additional layer of Kimaki, which gives you Discord integration (hate it or love it). Chatting with your codebase (voice messages, too, if that’s your jam), is very very cool.

Re: My Homelab AI Dev Platform

#10

Post by sosodev » Mon, Jun 15, 2026, 4:49 PM UTC

rsgm wrote: post #7

fazgha wrote: post #4

So first post in the blog, and it went directly HN frontpage. Then, I said homelab AI, I thought it's an interesting post about local GPU setup (and I am really interested in this topic).. but no, just another hype post about how to use whatever-code...

I looked into running local models last month. They just aren't quite there for agentic tool use workflows without spending a small fortune. I'm hopeful smaller local models get much better soon. I was also hoping to put out another post on my homelab setup, it has some neat stuff, but I haven't had a chance to finish it.

I think it heavily depends on what you're asking the model to do. Qwen3.6, both 27B and 35B-A3B, do agentic tool use very well. Their decision making is sus, but the dense model is decent in that way. A 4-bit quant for either of those can run on many home systems with a bit of configuration.

The biggest issue I've noticed is that the chat templates for open models are really hit or miss. The default Qwen3.6 chat template mostly works these days, but depending on your workload it may cause major issues. There are plenty of "fixed" chat templates on hugging face, but people report mixed success. It really seems to depend a lot on what the tool you're using expects.