Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

sunkcost.ai

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#31

Post by throwaway894345 »

txrx0000 wrote: post #8

It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…

I’m very sympathetic to this point of view but I also can’t remotely afford the hardware required to get in the ballpark of Fable.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#32

Post by taraindara »

gruez wrote: post #9
hyperhello wrote: post #2

I doubt it will ever be cost effective for the foreseeable future. The AI companies have astonishing amounts of compute and they’re effectively dumping it on the market.

"If they are selling it for less than it cost to make, buy as much as you can." -- Warren Buffett

Only caveat is you’re buying time. Not a physical good. It’s only worth what you’re able to get out of it in that time.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#33

Post by rlindsey123 »

gruez wrote: post #9
hyperhello wrote: post #2

I doubt it will ever be cost effective for the foreseeable future. The AI companies have astonishing amounts of compute and they’re effectively dumping it on the market.

"If they are selling it for less than it cost to make, buy as much as you can." -- Warren Buffett

Is that a real quote? Golden if true

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#35

Post by koito17 »

no-name-here wrote: post #13
txrx0000 wrote: post #8

It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…

You can't run recent openAI/Anthropic models locally anyway, so wouldn't a better comparison be a different provider running Qwen or similar model? As then you can also compare against the exact model you'd have locally and any different data privacy of that particular provider ?

GP's point is about "sending tokens to someone else's computer" versus "keeping the tokens locally". I think model capabilities are secondary.

In May of this year, I was running qwen3.6:35b-a3b on my MacBook (bought in 2024). Obviously not as fast as, say, running a model on Cerebras, but a year ago it wasn't really feasible to have a local model running on my 2024 laptop with vision support. (Concretely, I was passing apartment diagram pictures to Qwen and making it compare different apartments for which ones would feel the most spacious while optimizing for initial moving costs and other factors.)

This was back in May and I wouldn't be surprised if there have been significant improvements since then.

Overall, I think it's fair to compare a workflow like "use llama.cpp locally to upload some pictures and ask questions" to "open the ChatGPT app, upload pictures from your phone, and ask questions". Sure, you can't run a model like GPT-5.4 locally, but the model is mostly an implementation detail here. What a user will care about is: "when I go with the llama.cpp option, am I getting useful information from my conversations?"

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#36

Post by octoberfranklin »

tyre wrote: post #24
txrx0000 wrote: post #8

It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…

I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…

I'm curious what people are sending to Claude that is so secret.

The proof to the Navier-Stokes problem.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#38

Post by truncate »

tyre wrote: post #24
txrx0000 wrote: post #8

It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…

I'm curious what people are sending to Claude that is so secret. Claude knows about my interior decorating, questions about light bulbs, curiosity about what the Galactic Empire was even trying to do, unpacking SCOTUS decisions, shoe trees, Fed inflation history, etc. What part of my brain is contained here? Sure, the conversations have back and forth (some have dozens of exchanges), but, like, that's not the secret…

>> what people are sending to Claude that is so secret

Its the same point used against privacy. What's so secret you are doing that you need privacy. I think in the end, its about privacy and not trusting these model companies with your data. Facebook manipulated people behaviors with all the data they had, no reason AI companies wont someday decide to do that same, and they have far more intimate knowledge.

When it comes to coding, I also don't like the idea of them taking my money and potentially at same time potentially using as dataset generator.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#39

Post by rlindsey123 »

monksy wrote: post #25

I wish you could put different setups on here. I have a couple of A6000s on an AM5.

I'm keen to add a way for people to add community based reporting which would allow this. Would you want to see anything else on the dropdowns to be able to enter your data on?

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#40

Post by jerf »

ChickeNES wrote: post #16
chasd00 wrote: post #11

Not a fair comparison really. If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. That has value a subscription does not. Idk about the quality of this setup but just pasting it here as an example. https://explainx.ai/blog/heretic-llm-abliteration-guide-2026

> If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. When does the average person actually need to do that?

We've already seen frontier models refuse to answer almost any question that touches on computer security and be very likely to kick out biology and chemistry questions even if they aren't all that close to breeding dangerous viruses or making explosives.

I expect this is only going to get worse. "Censorship" isn't just going to be about who you vote for and which political party the model will say nice things about and which it is more likely to say bad things about. It's going to become about whether the hoi polloi are allowed to have effective AIs at all. Like the 1990s internet, AI has outrun a lot of power structures but that is not going to continue indefinitely.

Post reply on HN