Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an (almost) fully self-hosted, sandboxed, agentic software factory

blog.jakesaunders.dev

Post reply on HN
Print view View original

21–30 of 75 posts

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#21

Post by Philpax »

RajT88 wrote: post #2

So - apparently it's not fully self-hosted, since I don't see a GPU. I'm interested in hearing from folks who are hosting their own GPU to run coding models. So far my own results are... not great. Seems like frontier models are needed via the big providers?

Using a RTX 5090 ($3~4k), I can run Qwen 3.8 27B at ~180 TPS with ninfer [0]. With its thinking maxed out, I can confirm that the quality of output is roughly on par with Opus 4.5~4.6 - that is, this 20GB file really can write software by itself, but the amount of thinking required makes it strictly slower than larger models, even at 180 TPS.

Still, it's largely replaced the cheap tier of the frontiers that I would otherwise be using. It can be run with older GPUs (a 3090 is ~1k), but the time spent thinking will become a fairly noticeable impediment for staying in the flow.

The next step up would be to run DeepSeek V4 Flash 0731 on two DGX Sparks (~$10k), which serve at 60 TPS and sit somewhere around Opus 4.7 level without 3.8-tier thinking.

However, it is worth noting that, if you are buying this hardware just to serve LLMs, it is not cost-effective. It would take over a decade of continuous use to make back the cost of the DGX Spark setup in 0731 tokens. I'm running this setup because I happen to have a 5090, and the two 3090s in my server were cheap enough when amortised over several years.

[0]: https://github.com/Neroued/ninfer

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#23

Post by iforgetti »

primitivesuave wrote: post #20
RajT88 wrote: post #2

So - apparently it's not fully self-hosted, since I don't see a GPU. I'm interested in hearing from folks who are hosting their own GPU to run coding models. So far my own results are... not great. Seems like frontier models are needed via the big providers?

My favorite real-world example that I worked on: I created a YouTube documentary series about corruption in a small town in Illinois. This required downloading thousands of hours of government meeting videos from YouTube, transcribing + chunking + embedding, summarizing meeting segments, running a couple passes of validation, then searching for interesting storylines. I also scraped thousands of public documents whic…

Sounds amazing! What’s the name of the series if we wanted to find it on YouTube?

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#24

Post by tcdent »

copemaxxxing wrote: post #19

People who build software factory are in for a disappointing future. Spoken as someone who uses AI day to day for job and personal projects, I've came across a few super gnarly bugs already, ranging from frontend React apps (yes, believe it or not, frontend is far from solved) to embedded that Claude can't solve. Those bugs are feature breaker. My primary skillset is fullstack/frontend leaning, not embedded. Granted,…

What most people leave out of the conversation is the element of time. We are early. The technology continues to improve.

Is it perfect now? Absolutely not. Will it continue to get better at a surprising pace? Absolutely yes.

So, just adjust your framing from "we're there" to "we're getting there" and all of your concerns go away.

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#26

Post by mimischi »

deweywsu wrote: post #22

Why is it always assumed the reader knows the function of 15 obscure libraries and/or frameworks when posting projects like this?

I can relate to this feeling, having encountered it many times in academia. But the inverse would also be true, would you stop reading yet another article that explains for the umpteenth time what library Y does? Not arguing for the author, but there’s certainly something about knowing and writing for your audience.

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#27

Post by ashu1461 »

In such systems, producing code is the easy part, verification is hard. Verification via test cases just feels like the same agent validating its own assumptions.

Wondering what the actual verification loop looks like once you start taking these systems to production.

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#28

Post by ashu1461 »

deweywsu wrote: post #22

Why is it always assumed the reader knows the function of 15 obscure libraries and/or frameworks when posting projects like this?

I don't know most of these tools either, but that is why I find posts like this useful. Don't you end up learning something new in cases like this?

Re: Building an (almost) fully self-hosted, sandboxed, agentic software factory

#30

Post by bgilroy26 »

ashu1461 wrote: post #27

In such systems, producing code is the easy part, verification is hard. Verification via test cases just feels like the same agent validating its own assumptions. Wondering what the actual verification loop looks like once you start taking these systems to production.

It's really funny. I would read HN headlines about PhD research in the area of https://en.wikipedia.org/wiki/Correctness_(computer_science) and it always seemed really boring

Now it is obvious to me how important this work is

Post reply on HN