I want the autonomy but local models of the size I would have the means to host wouldn't be capable enough. What usecases tend to suit these smaller models that tend to produce incorrect or otherwise flawed responses often? Could they work for anomaly detection and what would a rough architecture look like?
Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
71–80 of 91 posts
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#72The idea that you need a new machine is pretty ridiculous. I bought a used HP Omen with a 3090 last month for $2k. 57t/s with Qwen 3.8.
I'm so happy for the two used 3090s I bought for $500 each after Ethereum mining ended. I even saw them for like $430 at some point lol.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#73It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…
I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…
Anthropic's goal is to commoditize intelligence. People who use their brains / intelligence for competitive advantage might not want to contribute training data for that goal.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#74Fun feature: can you show some sort of list of the best combos? Eg shortest payoff time for best capability in various situations.
Good idea, pretty crude but it's up: https://sunkcost.ai/best/
For each usage level, it lists the quickest pay-back in each capability class, with each model on its quickest machine and one click into the calculator to change the assumptions. Short version: at 1M tokens/day the best Sonnet-class option is Qwen3.8 27B on a Mac mini M6, 8.3 years. It only drops under a year if you're running agents at around 20M tokens/day.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#75It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…
I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…
> I'm curious what people are sending to Claude that is so secret.
When Claude is used in a professional setting, any or all of:
Proprietary intellectual property (a.k.a. system code)
PII[0] of the employee, customers, or both
HIPAA[1] data known to a system
Internal communications not meant to be publicized
Sensitive data, such as SSH keys and the like
0 - https://en.wikipedia.org/wiki/Personal_data
1 - https://en.wikipedia.org/wiki/Health_Insurance_Portability_a...
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#76Earlier quoted context omitted.
I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…
Anthropic's goal is to commoditize intelligence. People who use their brains / intelligence for competitive advantage might not want to contribute training data for that goal.
> Anthropic's goal is to commoditize intelligence.
And Google's original goal was to organize the world's information.
How did that turn out?
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#77Not a fair comparison really. If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. That has value a subscription does not. Idk about the quality of this setup but just pasting it here as an example. https://explainx.ai/blog/heretic-llm-abliteration-guide-2026
That can also be done with neoclouds.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#78This is a bit weird because it automatically changes the model depending on the amount of VRAM available, and some of the smaller models are more expensive (presumably because they're being provided via OpenRouter by someone with some GPUs in a colo or smaller providers). It also doesn't allow changing the tokens per second (my 5090 can get like 75-130 tokens per second [assuming I can fit the model in RAM]); which, then results in woefully under-estimated limit on how many tokens a day I can consume.
Some improvements that I think would make this more useful:
1. Allow manually setting tokens per second, or as an alternative, let me jack up the number of tokens a day.
2. A sort of backwards flow "if you want to run this, at X tokens per second, with Y context, you'd have to spend Z."
3. Add support for configuring multiple RTX 6000 variants.
When I was making heavy use of DeepSeekV4-pro I was burning somewhere around 1.5 billion tokens a month, and that was just using it in my free time on random projects. It was something like $24 at the time because of the initial discount/promo period. I don't think there's anyway in hell I could ever run that (on current hardware) for less money.
I think the calculator shows from a purely financial standpoint what we all know... that yeah, it's definitely not worth it if money is your only concern. That (cost per token) will eventually change. Models will get better, more efficient, VRAM prices will come down, VRAM capacity will rocket upwards, and the economics of it all will change. It would just be really cool to have the calculator show me exactly how cheap they'd have to get for it to make sense.
I need to finish up some work and make dinner, and if no one else beats me to it (anyone is welcome to) I'll ask Fable or Opus to knock that out.
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#79It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…
I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…
Are you willing to bet that your lack of imagination for exploitation is precisely that of several multibillion dollar companies?
Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
#80I have a home server running vibed applications. VPS host would cost $25/mo or $300/yr. Mac mini can also build iOS applications. I think if you’re a mobile dev, you can have concurrent builds for your agents instead of everyone waiting on a single machine to finish.
What models are you running on it? I'm also an iOS dev but I find I need more frontier models to get good quality code from it.
I only use frontier models to vibe code iOS apps, as I'm not an iOS developer. I haven't tried the local models post qwen coder 3.5 release for all the reasons.
AFAIK, a limiter for iOS engineers (and AI agents) for concurrent feature development is the xcode environment and hardware limits. BE engineers can easily have 3 agents working on 3 different microservices (or gitwork trees), but iOS devs can basically only manage one version of the code at a time, due to externalized state (like derived data and bundle ids).