Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI coding at home without going broke

stephen.bochinski.dev

Post reply on HN
Print view View original

81–90 of 321 posts

Re: AI coding at home without going broke

#81

Post by ajbourg »

baq wrote: post #59
theossuary wrote: post #49

Earlier quoted context omitted.

I truly think by 2028 we'll have integrated chip systems that'll be able to run opus 4.8 level models at ~500 watts at acceptable performance. Honestly I think now is the worst time to invest in AI hardware. Get your harness ready and processes perfected with hosted models, and wait a few years to buy hardware to transition to running models locally

Burning weights onto a chip in an efficient way and exposing that via USB would be acceptable for a good enough model tbh

This is pretty close to what Taalas is doing.

Re: AI coding at home without going broke

#82

Post by morganastra »

mwcampbell wrote: post #33

I invested about $4,000 in an NVIDIA DGX Spark several months ago. 128 GB of unified RAM, and the NVIDIA GB10 chip. With the RAM, the several CPU cores, and the 4 TB NVMe SSD, it's a very capable ARM64 Linux computer even without the GPU, and so far I've mostly been using it as such. But I wonder, what's the most capable model, specifically for coding, that can run well on that hardware?

Deepseek v4 flash is shockingly strong for its size and reportedly runs well on that hardware.

Re: AI coding at home without going broke

#83

Post by geophph »

> Do that well and you can build what a team of twenty engineers would put out in a month for around a thousand dollars.

What does this look like after 6-12 months? Like, how much code are you trying to write total?

Maybe it just doesn’t click in my mind, but sometimes I wonder about how much work people are trying to do and how they actually have enough to get done so quickly in such a short amount of time.

Re: AI coding at home without going broke

#84

Post by contingencies »

sheremetyev wrote: post #77
tunesmith wrote: post #47

I feel like I must have plateued and don't know what to do next to level up. I'm currently on the $100/month codex plan and it seems fine using 5.5-xhigh all the time. I think of what to do next, have a chat session to determine exactly what to ask for up to the point of being ready to implement, and then codex churns on a commit-sized task whereupon I briefly check it on my local dev server. If necessary I ask for a…

> I don't want to give it "dangerous" access to my entire mac I'm running Claude/Codex inside native macOS sandbox, configured with a simple script - https://github.com/sheremetyev/sandfence always in "bypass permissions" mode - it works until task is solved, sometime 1 hour or more (which includes running tests etc)

recommend converting to https://github.com/apple/container

Re: AI coding at home without going broke

#85

Post by PeterStuer »

tunesmith wrote: post #47

I feel like I must have plateued and don't know what to do next to level up. I'm currently on the $100/month codex plan and it seems fine using 5.5-xhigh all the time. I think of what to do next, have a chat session to determine exactly what to ask for up to the point of being ready to implement, and then codex churns on a commit-sized task whereupon I briefly check it on my local dev server. If necessary I ask for a…

I'm on $100 Claude. I have a setup with bespoke local services that mitigates some high token consumption scenarios with local LAN services. I screen mcp's and hooks for cache poisoning. I run 100% on Opus with max effort, and never came close to hitting 5 hour or weekly limits before the Fable release. I am in Claude Code at least 20hrs a week.

I see people just completely wasting tokens with ridiculous setups, 100% hitting cache misses as well as dumping huge files into context all the time.

Just learn how these things work, or pay the price I guess.

Re: AI coding at home without going broke

#86

Post by simonw »

jacobgold wrote: post #73
simonw wrote: post #55

Earlier quoted context omitted.

SemiAnalysis pushed this to the limit and managed to get $8,000 of tokens from a $200/month Anthropic plan and $14,000 of tokens from a $200/month OpenAI plan: https://twitter.com/SemiAnalysis_/status/2064815044085318040

Yeah, although that is pushing every rate limit and no one knows what happens if you do that consistently? I think $4,000/mo is probably a good estimate for an individual dev doing synchronous coding agent work.

Yeah, I agree. I've been consistently getting about $1,000/month of value out of the $100/month subscription for OpenAI, and about the same for Anthropic.

Re: AI coding at home without going broke

#87

Post by hurtigioll »

theossuary wrote: post #49
Catloafdev wrote: post #27

Earlier quoted context omitted.

If you want frontier-level, the economically reasonable option is OpenRouter or a direct sub to frontier-of-your-choice. The reality is that they do not offer configurations that would allow a consumer to run that much VRAM on a single setup to protect datacenter margins. Apple used to, and they stopped, those devices are going for ~$20k+ each on ebay now. You can get very, very capable models on a 3090/4090/5090/600…

I truly think by 2028 we'll have integrated chip systems that'll be able to run opus 4.8 level models at ~500 watts at acceptable performance. Honestly I think now is the worst time to invest in AI hardware. Get your harness ready and processes perfected with hosted models, and wait a few years to buy hardware to transition to running models locally

if such hardware becomes available, it will be bought by the data-centers, just like they buy all the RAM today

Re: AI coding at home without going broke

#88

Post by hillj23 »

I think this is only going to become more relevant. I'm personally a $200/mo Claude Maxer and I know that the usage I'm getting on Opus 4.8 Max and (until they yoked it out from under me) Fable 5 is way, way more than what I'm paying them. At some point, this will turn usage-based and I will be hammered on it and probably forced to look at self-hosting. I think while the caps are there, even at $200, it's honestly not too bad if you're coding value into the market, but as soon as those caps come off for retail AI users, we're all going to have some tough choices to make.

Re: AI coding at home without going broke

#89

Post by bachmeier »

> The upfront cost is steep and the models you can actually run at home are weaker than what the frontier labs ship, so this only pays off if you can keep the rig busy with long running tasks where a slower, cheaper model grinds away overnight. Most people can’t keep a home machine that loaded, and the hardware you buy today may look like a bad bet in a year.

Oh, so this is not a post about AI coding at home. It's about vibe coding at home.

There's a lot I disagree with in this post, but I'm posting this from a home computer with 64 GB of RAM and no GPU. I do lots of AI coding while spending very little money. I run Gemma 4 26b (mixture of experts) and Qwen 3 coder with Ollama. I use Github Copilot code completions. I use the Gemini and Mistral API free tiers. I have a Gemini paid API account. It's now prepaid, so you don't have to worry about an accidental $1000 bill. You can do a lot of things with Gemini Flash Lite 3.1.

None of this is burning through tokens to create an expensive blob of spaghetti code, but it does qualify as AI coding.

Re: AI coding at home without going broke

#90

Post by sheremetyev »

contingencies wrote: post #84
sheremetyev wrote: post #77

Earlier quoted context omitted.

> I don't want to give it "dangerous" access to my entire mac I'm running Claude/Codex inside native macOS sandbox, configured with a simple script - https://github.com/sheremetyev/sandfence always in "bypass permissions" mode - it works until task is solved, sometime 1 hour or more (which includes running tests etc)

recommend converting to https://github.com/apple/container

Linux VM doesn't run native macOS toolchain and requires copying files back and forth
Post reply on HN