The unbearable cheapness of open weight models • Hacker News Board

Re: The unbearable cheapness of open weight models

#81

Post by crazylogger » Thu, Jun 25, 2026, 2:37 PM UTC

pjc50 wrote: post #54

arikrahman wrote: post #6

With cache hit rates being effectively free, harnesses like Reasonix have let me do a month of work for less than 2 dollars. It's not even the subsidies making it cheap, American providers like Digital Ocean or Cloudflare host the same model with similar pricing.

How does caching help here? How much repetition is there in queries?

In a typical agent loop your N-th LLM request naturally becomes prefix for the (N+1)-th request. As the thread grows longer, cache hit rate converges to 100% and unit pricing for cached tokens is 10-100x cheaper.

Re: The unbearable cheapness of open weight models

#82

Post by samuelknight » Thu, Jun 25, 2026, 3:11 PM UTC

fredley wrote: post #23

Jackobrien wrote: post #7

The giants knew this was coming, and soon 95% of AI tasks will be able to be done by open models (coding, research, cowork style work). So why pay a premium? Why use them at all? This leaves the labs with two options: 1) push the frontier in a way only massive scale can, and cash in on it (mythos level cyber security, recursive training, frontier science work). There’s big money for never before possible capabilities…

3) Buy all the RAM, increasing the barrier to entry to push back the tide a bit, in time for a juicy IPO.

Buying all the RAM can't work forever. Scarcity increases prices, high prices increase supply, improves RAM R&D budgets, and forces users to find ways to economize around low RAM availability.

Re: The unbearable cheapness of open weight models

#83

Post by titanomachy » Thu, Jun 25, 2026, 3:24 PM UTC

isoprophlex wrote: post #62

Aren't these open models so cheap because they're (partially) chinese gov. sponsored, and because they're stealing and redistributing the IP that comes in?

Maybe, but there's tons of providers available, so you can pick one that you trust not to steal your IP (or run it yourself, if you're rich and paranoid enough).

Re: The unbearable cheapness of open weight models

#84

Post by saltcured » Thu, Jun 25, 2026, 4:21 PM UTC

NitpickLawyer wrote: post #19

ed_elliott_asc wrote: post #11

Earlier quoted context omitted.

Won’t all they need to do is say “best in class, latest models, fastest” and wine and dine a few execs and those enterprise deals will be signed? In this case the people tasked with using the product won’t actually mind.

No one is getting fired for using SotA.

Well, getting laid off during the bankruptcy spiral is a form of firing.

But that is months away, so not my problem?

Re: The unbearable cheapness of open weight models

#85

Post by kyleomalley » Thu, Jun 25, 2026, 4:37 PM UTC

RugnirViking wrote: post #55

orwin wrote: post #51

Earlier quoted context omitted.

Mythos was outperformed by small, specific local models in multiple oss project.

i'd love to hear about this! do you have examples?

It might be kind of overlooked when people read about the big scary results from mythos; the real breakthrough was probably just as much the application of the (very decent) model through a well engineered wrapper (harness). Other models including codex or glm result in significant findings as well.

Harness example: https://github.com/evilsocket/audit

Re: The unbearable cheapness of open weight models

#86

Post by arthurofbabylon » Thu, Jun 25, 2026, 4:47 PM UTC

AnthonyMouse wrote: post #58

arthurofbabylon wrote: post #10

Let's imagine that Anthropic/OpenAI fail to manufacture scarcity by villainizing Open Weight models (a sincere probability). What is left for these corporations to prop up their prices, or any margin at all? I expect scaffolding around tool use, supporting bespoke implementation and driving risk down for institutional adoption. (They might even build an insurance tool to protect accountants/lawyers from errors in com…

> It seems plainly clear to me that information and information processing is commodifying (for the first time in human history?). Without the age-old bottlenecks at the top of the value chain, capital will surely flow downwards, right? Isn't this the thing people have said about every new technology since the printing press? And it has been mostly true, but it has also been the case that the incumbents have fought h…

I don’t think that comparing LLM’s to the printing press (and radio, film, TV, etc) is an apt analogy, and I don’t think that people have said the same things about the two technologies; the prior technological changes in information dealt with distribution, while this one deals with processing and production.

Recall the notion of a bottleneck, and this distinction will become clear. Those prior technological changes never inverted a bottleneck, and this one does.

Re: The unbearable cheapness of open weight models

#87

Post by beepdyboop » Thu, Jun 25, 2026, 4:56 PM UTC

I don’t get it. So many here are saying open weight models will kill the frontier labs. But open source and similar have tried to beat private companies everywhere all the time, and people still buy the best products even if great open source alternatives are available. Why wouldn’t this be the case for AI too?

Re: The unbearable cheapness of open weight models

#88

Post by OkayPhysicist » Thu, Jun 25, 2026, 5:01 PM UTC

samuelknight wrote: post #82

fredley wrote: post #23

Earlier quoted context omitted.

3) Buy all the RAM, increasing the barrier to entry to push back the tide a bit, in time for a juicy IPO.

Buying all the RAM can't work forever. Scarcity increases prices, high prices increase supply, improves RAM R&D budgets, and forces users to find ways to economize around low RAM availability.

It doesn't need to work forever. You just need to delay your competitors long enough that you can IPO to great fanfare, and then leave retail investors holding the bag. Founders and big investors get to cash out, everyone else gets screwed.

Re: The unbearable cheapness of open weight models

#89

Post by drudolph914 » Thu, Jun 25, 2026, 5:06 PM UTC

beepdyboop wrote: post #87

I don’t get it. So many here are saying open weight models will kill the frontier labs. But open source and similar have tried to beat private companies everywhere all the time, and people still buy the best products even if great open source alternatives are available. Why wouldn’t this be the case for AI too?

I feel like this comment is just engagement farming, but I'll bite anyways

there is a larger appetite for something like open source AI mostly b/c of price. we all know these labs have not figured out their pricing model, and we're all holding our breath out of fear of what the prices could be.

also, if you consider that the only toll to knowledge work before was personal time, and now you need to pay $100s month just to keep up with the baseline speed. it makes sense people are looking for something that gets them back to a workflow where the price to do work is near $0.00.

I think for a smaller group though, it's more to do with a certain combination of principles. Some people don't want censorship, other's want ownership, some want the knowledge of working on LLMs to not be gate kept.

Re: The unbearable cheapness of open weight models

#90

Post by kittikitti » Thu, Jun 25, 2026, 5:07 PM UTC

Even if open weight models were vastly more expensive, I would still prefer them. I don't know where my data is going and whether they're lying about the model when I make an API call. They can ban you from their API for any reason. Anthropic recently pulled their frontier models. There are numerous compliance concerns. The list goes on and on.