It pays for itself very quickly if you do 24/7 generation. Use an AI agent that orchestrates other agents working on many things at once constantly. If speed is a factor, you'd not buy a Macbook, you'd buy dual RTX 3090s. About the same price, but at least 6x faster than M5 Max. The benefit of constant generation is you can do a lot more research, coding sub-agents, experiments, etc in parallel when you're not "at work". You end up getting a lot more work done than if you only sit there babysitting sessions.
Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
61–70 of 91 posts
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#62Earlier quoted context omitted.
> If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. When does the average person actually need to do that?
"Need" might be a bit too strong, but I do want overly obnoxious guardrails not to stand in the way. Case in point, last week I was poking Opus 5 into writing me some RPi-pico firmware for driving a small e-paper screen. Font was built in right into C code as hex constants. Space being tight, I asked if there is some clever compression that could be applied. Claude thought for good 10 minutes, then guardrail kicked i…
Just apply to the cyber program? I got in in around 2 hours, and I'm just some hobby hacker, not some paid security consultant. It is a valid point though, I'm doing tons of systems and embedded stuff and was hitting the safe guards with Claude and Codex before getting into their cyber programs (hex REALLY triggered Claude in particular, which was amusing).
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#63Earlier quoted context omitted.
We've already seen frontier models refuse to answer almost any question that touches on computer security and be very likely to kick out biology and chemistry questions even if they aren't all that close to breeding dangerous viruses or making explosives. I expect this is only going to get worse. "Censorship" isn't just going to be about who you vote for and which political party the model will say nice things about…
So you want to remove valid safeguards? And stop misusing the word censorship.
You asked a question. I gave you an answer. I seriously doubt that if you and I sat down together at a table and banged on this for an hour that we would come to the same definition of "valid". Ask 10 people, get 12 answers to that question. There's going to be a lot of motte & bailey in the next couple of years, where I just want an AI to answer questions about whether my code is vulnerable and people like you will be "Oh so you want an AI that can hack the Pentagon do you?" and it doesn't look like we're going to be seeing eye to eye on that one.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#64The math is wrong, the tok/s is at least 2x that, at least with MTP and Q8 KV which you should always use. And the default tokens a day is ridiculously low at least for coding.
Having said that, it will never pay for itself. A simpler more absolute math is, if I buy a Mac and use it to sell tokens on OpenRouter, will I make a profit? And the answer is no.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#65Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#66Besides from privacy: I already making twice now.you own the hardware and the price had doubled since i bought. Almost tripled. You missed the opportunity and i have 4 of those awesome machines. Cry on. I sell those to business who need local air gapped requirments and I make a lot more money! I can run the alliterated models where none of the service prvoider even dare to provide. THose benefits outweights a few K.…
> And show me an api provider that allows me to run 10x agents concurrently for 5 days straights .
Any of them on a Max/Pro plan as long as you are smart about model selection? That's my main objection to local inference, I'd need a whole rack of GPUs to do as many things in parallel that I can do for $400 a month. I do plan on setting up some local inference hardware, but...RAM and GPU prices alone are $$$$
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#67I doubt it will ever be cost effective for the foreseeable future. The AI companies have astonishing amounts of compute and they’re effectively dumping it on the market.
More than that, running hundreds of conversation streams at once is essentially the same cost as running a single conversation. And then you add on the secondary benefit of having the GPUs running nearly all the time rather than mostly idle...
Local inference makes sense for speciality needs, or very small models. But if your model is bug enough to span GPUs its excessively wasteful to hoard those GPUs for yourself without piggybacking hundreds of other conversations on top of all that memory bandwidth and matrix multiplies.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#68Earlier quoted context omitted.
You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you can also compare against the exact model you'd have locally and any different data privacy of that particular provider ?
GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary. In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passi…
[deleted]
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#69Earlier quoted context omitted.
I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…
I'm curious what people are sending to Claude that is so secret. The proof to the Navier-Stokes problem.
I'm pretty sure they were sending a prompt for Claude to _find_ the Navier-Stokes proof, using ideas that have been publicly shared before online, but not necessarily used for the problem.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#70Earlier quoted context omitted.
You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you can also compare against the exact model you'd have locally and any different data privacy of that particular provider ?
GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary. In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passi…
Deepseek 4 flash can run locally , and qwen 3.8-next-flash , they are already gpt 5.6 tier.