Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

sunkcost.ai

Post reply on HN
Print view View original

101–110 of 111 posts

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#101

Post by shadowpho »

rlindsey123 wrote: post #21
shadowpho wrote: post #10

I like this calculator but it’s really wrong at least for dgx spark. I have one and I get 4x the tokens/s .

Aw very interesting! This is great feedback - what model are you running? I'm keen to do more crowdsourced data as time goes on.

The big three :)

Qwen3.8-flash-next Deepseek4-0731-flash Glm5.3

The latest unsloth llama.cpp has a lot of nice features that runs them faster than before.

I’ll have to double check which one runs how fast, but it’s generally 20-40 t/s. (And infil is fast but not sure how that’s counted)

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#102

Post by ASalazarMX »

txrx0000 wrote: post #8

It pays off instantly , because OpenAI/Anthropic can no longer see what I'm doing and that's worth a lot of money to me. If I am offloading some of my thought processes to a machine, I want to own that machine. And if I finetune the model, I can gain access to parts of thought space that are cordoned off by OpenAI/Anthropic/Alibaba/whomever due to their "alignment" efforts (i.e. alignment to the AI company rather tha…

> If I am offloading some of my thought processes to a machine

"Offloading thought" sounds a lot better than "outsourcing thought", but the latter is what we're really doing. Offloading implies you thought it first and then gave it to the LLM, but we're only giving it the minimun so it can do most of the work in our place,

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#103

Post by ASalazarMX »

hyperhello wrote: post #2

I doubt it will ever be cost effective for the foreseeable future. The AI companies have astonishing amounts of compute and they’re effectively dumping it on the market.

The comparison is about how many tokens you buy vs how much hardware you could buy with the same money. It's as saying "if you have rib eyes at Applebee's every day, how long until cooking your own rib eyes pays for itself".

If you don't consume many of tokens, it will likely never pay for itself. If you do, though, it will have trade-offs, but you'll probably save money in the end.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#104

Post by ASalazarMX »

ChickeNES wrote: post #16
chasd00 wrote: post #11

Not a fair comparison really. If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. That has value a subscription does not. Idk about the quality of this setup but just pasting it here as an example. https://explainx.ai/blog/heretic-llm-abliteration-guide-2026

> If you can run a model locally then you can somewhat train out the guardrails, censorship, and brand-safety. When does the average person actually need to do that?

If you mean retraining, it's not even needed anymore. If you want the guardrails off, these days you just install LMStudio and download an abridged model. It's all GUI. The abridged models might have weird behavior in edge cases after the pruning, though.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#105

Post by LargoLasskhyfv »

v3ss0n wrote: post #95
LargoLasskhyfv wrote: post #85

Earlier quoted context omitted.

Regarding DeepSeek, which I also like very much, have you tried https://reasonix.io ?

Bot? Care to explain any difference vs DSH / OpenCode / Hermes ?

Notbot! Are you unable to read, or what? Just fkn install it, and see how smooth it integrates?

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#107

Post by ThunderSizzle »

rickydroll wrote: post #99
ThunderSizzle wrote: post #12

Claude Code is $100+ or else be constantly throttled. My usage on GHCP was gonna be $300+ a month. I paid $1350 and threw an R9700 in an existing machine. That's a 4 month pay off or so. Plus, I can feed it sensitive data all day and not be worried where it's going.

today, the r9700 is 1800$ at micro center :-( so payoff is around 2 months?

Sadly. It's jumped $300-400 in 2-3 months time.

I don't think I'm the only persona that did the math.

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#108

Post by ThunderSizzle »

no-name-here wrote: post #17
ThunderSizzle wrote: post #12

Claude Code is $100+ or else be constantly throttled. My usage on GHCP was gonna be $300+ a month. I paid $1350 and threw an R9700 in an existing machine. That's a 4 month pay off or so. Plus, I can feed it sensitive data all day and not be worried where it's going.

An R9700 has 32 GB RAM. Is your comparison against a similar size model? Or shouldn't you be comparing it against the cost of a hosted model matching the one you’re using locally?

Well, I don't see a value issue of using Qwen3.6 27B vs Sonnet 4.6 (not sure about 5 yet)

I still have to use GHCP at work, and I self-host at home, and aside from the fact self-hosting also forces you to tinker, optimize, etc. - there's not a huge difference in my end result in end user results. I spent quite a bit of time trying to optimize llamacpp and compare 35b to 27b, etc. I don't compare models that much at work.

I guess the other part of it is I didn't really know much about cheaper cloud models, but I was attracted to the idea of no longer renting against Claude code, etc. I figured if I could run something functionaly similar from my bedroom on a normal outlet, then all this talk about data centers needing to be built everywhere in the news cycle is obviously just plain stupidity and hype.

It appears I'm using about 20.4/7.6 million in/out tokens a month, or on open router, about $20/month.

That puts $1350 at a 5-6 year break even (thanks to cheap electricity), I guess. Beyond that, running on localhost as a nice feature of 0 no latency when doing rapid tool calling

Re: Show HN: Sunk Cost – How long until a local LLM rig pays for itself?

#110

Post by ChickeNES »

v3ss0n wrote: post #91
ChickeNES wrote: post #66

Earlier quoted context omitted.

> And show me an api provider that allows me to run 10x agents concurrently for 5 days straights . Any of them on a Max/Pro plan as long as you are smart about model selection? That's my main objection to local inference, I'd need a whole rack of GPUs to do as many things in parallel that I can do for $400 a month. I do plan on setting up some local inference hardware, but...RAM and GPU prices alone are $$$$

> I'd need a whole rack of GPUs no , all you need is one small DGXSPark with proper setup.

You really don't understand how many agents I have running at once
Post reply on HN