51–60 of 321 posts
> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that. Power is not free. What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.
Earlier quoted context omitted.
What would you do for the rest of the day, power off your devices and go for a long bike ride?
Speaking personally: yes. That's literally what I'm planning to do this afternoon because it's noon and I'm already done with the coding tasks I had on my plate today.
> The first is to self host. You buy the machine, run open source models locally, and pay nothing per token after that. Power is not free. What I’ve found is that you’re basically paying a premium for privacy, and that’s worth it for me.
its ~free if you have home solar.
For me, investing in hardware seems to be the way to go. I learned coding nearly 24 years ago and still learning new stuff all the time. At no point in time I had to rely on a subscription model to learn and do new stuff. If LLM and agents are the default tools for coding and building software, at least for next few years, it seems like a no-brainer to invest $2000-3000 on hardware, like a Halo Strix PC.
"Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling." I realize this text is just slop but it never stops being a "real bargain" at any point. And it's more like $200/mo for $4000+/mo in tokens. You can also buy additional subscriptions. There's no sense in running local models or doing anything else as long as VCs (and soon the publ…
Can I run something comparable to Opus 4.6 locally yet? I keep hearing conflicting things. If I can spend 10k to do that I would cancel my subscription. The problem is I don’t wanna spend the money to find out myself.
"Around $400 a month of plans buys roughly $2800 of API usage at list prices, which is a real bargain right up until you hit the ceiling." I realize this text is just slop but it never stops being a "real bargain" at any point. And it's more like $200/mo for $4000+/mo in tokens. You can also buy additional subscriptions. There's no sense in running local models or doing anything else as long as VCs (and soon the publ…
Even if they were making a profit, their scale and expertise will obviously give you a cheaper product than what you can build.
At the end of the day, AI models are relatively small files that we run little CUDA programs on.
No clue what y'all are doing, perhaps because I'm hobbying, and also I'm old and can perhaps do more of this by hand.
But I'm basically just doing what I did before, plus ollama self hosted and sometimes gemini and I feel like I'm going lightspeed beyond what I've ever done.
And I suppose this is still very fine-grained. I have it make a draft, then just have them fix/change it step by step?
I tried one of the bigger boys that can one-shot apps, which I guess is cool, but I'm finding it's just as hard to modify as if I just grabbed someone elses repo on github.
Earlier quoted context omitted.
If you want frontier-level, the economically reasonable option is OpenRouter or a direct sub to frontier-of-your-choice. The reality is that they do not offer configurations that would allow a consumer to run that much VRAM on a single setup to protect datacenter margins. Apple used to, and they stopped, those devices are going for ~$20k+ each on ebay now. You can get very, very capable models on a 3090/4090/5090/600…
I truly think by 2028 we'll have integrated chip systems that'll be able to run opus 4.8 level models at ~500 watts at acceptable performance. Honestly I think now is the worst time to invest in AI hardware. Get your harness ready and processes perfected with hosted models, and wait a few years to buy hardware to transition to running models locally
Earlier quoted context omitted.
Actually if you have solar, it kind of is.. so prIvAt AI compute gets defacto cheaper during the day?
If you have solar, it is not, because you have battery and equipment degradation from cycle charging, c’mon man… I would agree with you if you said it was vastly cheaper overall (with the initial equipment investment amortized over time) compared to The Power Company. In many states, even if you are generating electricity and selling it back to the power company, they still gonna charge you normal rates of usage beca…