AI coding at home without going broke • Hacker News Board

Re: AI coding at home without going broke

#111

Post by zozbot234 » Sat, Jun 13, 2026, 7:21 PM UTC

evrydayhustling wrote: post #80

dofm wrote: post #17

Earlier quoted context omitted.

If an identical task takes a day on both sides, then the human route uses less energy, surely. Brains are thousands or maybe even millions of times more fuel-efficient than computers and you are alive for the whole day either way, right? You probably eat about the same even. The reason executives think AI is more efficient is that it more space efficient than a human and doesn't demand to be paid or work only a set n…

Brains are efficient, but civilized humans aren't. In the USA, adults consume at a rate of about 10kW -- only 1-2% of that being the human's metabolism, the rest being HVAC, electrical devices, etc. For comparison, a modern frontier model like Gemini 3.5 Pro consumes about 15kW -- so only about 1.5x the fully loaded human. In an 8h workday, that model would crank through ~80M tokens (~$5k at API prices). That's ~4 ma…

You forgot to mention that it takes a lot more energy to train that human before they're able to work.

Re: AI coding at home without going broke

#112

Post by [deleted] » Sat, Jun 13, 2026, 7:21 PM UTC

wincy wrote: post #104

tunesmith wrote: post #47

I feel like I must have plateued and don't know what to do next to level up. I'm currently on the $100/month codex plan and it seems fine using 5.5-xhigh all the time. I think of what to do next, have a chat session to determine exactly what to ask for up to the point of being ready to implement, and then codex churns on a commit-sized task whereupon I briefly check it on my local dev server. If necessary I ask for a…

I’m using $200 a month Codex working on a game for my kids for fun and curiosity since I’m a dev, I’ve played games, but I’ve never done dev for games. and have all night tasks but mostly they’re “spend time tending to and adding stuff to my 3D asset pipeline”. My RTX 5090 runs Trellis2 -> ultrashapes -> Trellis2 -> wiring up rigging and setting up animations. But like 99% of that task is just Codex waiting for the o…

[deleted]

Re: AI coding at home without going broke

#113

Post by calgoo » Sat, Jun 13, 2026, 7:22 PM UTC

ajbourg wrote: post #81

baq wrote: post #59

Earlier quoted context omitted.

Burning weights onto a chip in an efficient way and exposing that via USB would be acceptable for a good enough model tbh

This is pretty close to what Taalas is doing.

Trying Taalas is almost scary, there is something unsettling with that speed! Even with that small model, because of the speed, you could run hundreds of sample runs in a second, and pick from the best.

Can't wait for their next release!

Re: AI coding at home without going broke

#114

Post by 0xB0D » Sat, Jun 13, 2026, 7:26 PM UTC

If your job becomes writing complex specs to make an LLM write code, you've not optimised anything.

In fact all you've done is add a business cost.

Re: AI coding at home without going broke

#115

Post by zozbot234 » Sat, Jun 13, 2026, 7:28 PM UTC

daemonologist wrote: post #36

Catloafdev wrote: post #27

Earlier quoted context omitted.

If you want frontier-level, the economically reasonable option is OpenRouter or a direct sub to frontier-of-your-choice. The reality is that they do not offer configurations that would allow a consumer to run that much VRAM on a single setup to protect datacenter margins. Apple used to, and they stopped, those devices are going for ~$20k+ each on ebay now. You can get very, very capable models on a 3090/4090/5090/600…

There are also significant economies of scale (namely: utilization and batching), which tend to make inference on a shared server more economical even after the operator takes a cut.

You can use batching on consumer hardware, it just requires a KV-cache efficient model (or short context only) and keeping multiple inference flows running in parallel. This is most useful in combination with streamed inference, since the compute intensity of decode with those newer KV-compressed models is high enough that you have limited compute headroom when running at the speed of RAM.

Re: AI coding at home without going broke

#116

Post by zozbot234 » Sat, Jun 13, 2026, 7:32 PM UTC

CamperBob2 wrote: post #56

vadansky wrote: post #13

Can I run something comparable to Opus 4.6 locally yet? I keep hearing conflicting things. If I can spend 10k to do that I would cancel my subscription. The problem is I don’t wanna spend the money to find out myself.

Some benchmarks have shown Kimi K2.6 within error-bar distance of Opus 4.6, and you can run it on eight RTX6000s. Right now it's not possible to set up a machine like that from scratch for less than $100K... but right now it's also hard to put a price on autonomy.

You need a lot less than that if you're willing to stream the model from SSD. At that point, the best machine is probably a cheap old-gen HEDT with lots of PCIe lanes to attach cheap NVMe storage to, so as to stream the model at reasonable speed. That's expensive but not $100k expensive!

Re: AI coding at home without going broke

#117

Post by r0fl » Sat, Jun 13, 2026, 7:33 PM UTC

dnautics wrote: post #53

isatty wrote: post #3

> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that. Power is not free. What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.

> Power is not free. its ~free if you have home solar.

Solar panels are free and never break and have an unlimited life expectancy?

Re: AI coding at home without going broke

#118

Post by closeparen » Sat, Jun 13, 2026, 7:33 PM UTC

>Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling. The plans are metered, and any large AI native workflow will chew through the included tokens fast

I don't think that's true at all. I'm doing 8-12 PRs a week at work, all primarily Claude Code, and the usage at API billing has never broken $500/mo.

Re: AI coding at home without going broke

#119

Post by zozbot234 » Sat, Jun 13, 2026, 7:36 PM UTC

atemerev wrote: post #63

vadansky wrote: post #13

Can I run something comparable to Opus 4.6 locally yet? I keep hearing conflicting things. If I can spend 10k to do that I would cancel my subscription. The problem is I don’t wanna spend the money to find out myself.

Best you could do is connect two Mac Studio M3 Ultra 512G RAM each with Thunderbolt. Then theoretically you can run frontier Chinese models (but not Deepseek v4 Pro yet). That would be about $20k. But - good luck finding them. Apple discontinued the model a few months ago. And more recently, even 256G model was discontinued. Big AI really really does not want people to get off their needle.

DeepSeek V4 Pro is ~800GB total at native quantization (1.6T params with most being 4-bit) so it can run on the hardware you mentioned. There is also a 2-bit version that will run on a single 512GB machine. SSD streaming also makes lower-end hardware viable to at least test the model, if not quite run it usefully.

Re: AI coding at home without going broke

#120

Post by kgwgk » Sat, Jun 13, 2026, 7:36 PM UTC

closeparen wrote: post #118

>Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling. The plans are metered, and any large AI native workflow will chew through the included tokens fast I don't think that's true at all. I'm doing 8-12 PRs a week at work, all primarily Claude Code, and the usage at API billing has never broken $500/mo.

Maybe yours is not a large AI native workflow?