Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI coding at home without going broke

stephen.bochinski.dev

Post reply on HN
Print view View original

61–70 of 321 posts

Re: AI coding at home without going broke

#61

Post by dnautics »

reactordev wrote: post #26
warumdarum wrote: post #7

Earlier quoted context omitted.

Actually if you have solar, it kind of is.. so prIvAt AI compute gets defacto cheaper during the day?

If you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man… I would agree with you if you said it was vastly cheaper overall (with the initial equipment investment amortized over time) compared to The Power Company. In many states, even if you are generating electricity and selling it back to the power company, they still gonna charge you normal rates of usage beca…

> if you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man…

no, the rate of that is pretty independent of use. unless you live in a place where selling energy back rules are designed to screw the solar owner (California)

Re: AI coding at home without going broke

#62

Post by jrm4 »

isatty wrote: post #3

> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that. Power is not free. What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.

I'm in Florida and am already using AC, so if not "free", definitely "negligible."

Re: AI coding at home without going broke

#63

Post by atemerev »

vadansky wrote: post #13

Can I run something comparable to Opus 4.6 locally yet? I keep hearing conflicting things. If I can spend 10k to do that I would cancel my subscription. The problem is I don’t wanna spend the money to find out myself.

Best you could do is connect two Mac Studio M3 Ultra 512G RAM each with Thunderbolt. Then theoretically you can run frontier Chinese models (but not Deepseek v4 Pro yet). That would be about $20k.

But - good luck finding them. Apple discontinued the model a few months ago. And more recently, even 256G model was discontinued. Big AI really really does not want people to get off their needle.

Re: AI coding at home without going broke

#64

Post by WhiteOwlLion »

There’s a lot of Xeon chips for $10 on eBay. Too bad there’s no drive for cpu based inference. The data center will need to swap out the older gpu clusters so what does that do for hardware pricing on data center gpus? H100 are cheap enough but the power requirements make it a long term net negative for how much pay for power in California.

Re: AI coding at home without going broke

#66

Post by reactordev »

dnautics wrote: post #61
reactordev wrote: post #26

Earlier quoted context omitted.

If you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man… I would agree with you if you said it was vastly cheaper overall (with the initial equipment investment amortized over time) compared to The Power Company. In many states, even if you are generating electricity and selling it back to the power company, they still gonna charge you normal rates of usage beca…

> if you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man… no, the rate of that is pretty independent of use. unless you live in a place where selling energy back rules are designed to screw the solar owner (California)

California, Arizona, Texas, most of the southern states…

Re: AI coding at home without going broke

#67

Post by jrm4 »

esalman wrote: post #11

For me, investing in hardware seems to be the way to go. I learned coding nearly 24 years ago and still learning new stuff all the time. At no point in time I had to rely on a subscription model to learn and do new stuff. If LLM and agents are the default tools for coding and building software, at least for next few years, it seems like a no-brainer to invest $2000-3000 on hardware, like a Halo Strix PC.

With you here. I'm using my cheapo 16gig vram card I picked up a year or so ago, and I'm like -- yes, I percieve that you can pay for way more tokens per second that I can do at home.

But that feels like measuring productivity in lines of code. For what I'm doing, I'm not seeing the benefit in any subscription.

Sure, I can't one-prompt a whole new boring CRUD app, but oh well.

Re: AI coding at home without going broke

#68

Post by jrm4 »

pianopatrick wrote: post #14

I think someone could find some way to use the smaller local models to write code. Some kind of framework or harness or language or something. But not too many people are working on that because the big models are pretty cheap and a lot better.

I mean, this is what I'm doing. I'm guessing my process is very different because I'm holding the hand of the project way more along the way, but even that to me probably makes for a more enjoyable.

Which is to say, I might use AI to do an outline/organizational , but I'm prompting every chunk of code "one-by-one," (e.g. at about the "function" level) which still feels lightyears ahead of what I used to do.

Re: AI coding at home without going broke

#69

Post by throwaway219450 »

isatty wrote: post #3

> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that. Power is not free. What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.

Also, I would anticipate at least a 5 year lifespan for a current generation card. The 3090 is still respectable simply because it has 24GB of RAM which, for years, has been the limiting factor for ML at home. If you got a 6000, sure it’s going to cost 7-8k, but the resale value is likely to be very good. Even the 3090 is 50%+ of RRP still. And if you’re not doing LLMs, it’s an interesting value proposition for “classic” CNN vision model training. You can fit enormous batch sizes on 96 GB. The biggest reason to upgrade is perf/watt has about doubled (eg 4000 pro Blackwell is half the 3090 for similar).

People tend to assume the capex is thrown away but as we’ve seen with RAM, don’t be so sure you won’t be able flip it if you need to.

Re: AI coding at home without going broke

#70

Post by spgorbatiuk »

Hardware and provider juggling is a way to go, although I think it is also worth mentioning that the cost is not only the price-per-token, but first of all, the amount of tokens used.

Depending on what one builds, comprehensive documentation and applicable skills and memory tools often allow for a substantial reduction of tokens previously used by the agent to comprehend and remember what is being built

Post reply on HN