Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is minimal downside to switching to open models

marble.onl

Post reply on HN
Print view View original

41–50 of 351 posts

Re: There is minimal downside to switching to open models

#41

Post by bnj »

I’ve been wanting to get better acquainted with local inference but I don’t have the hardware, which has made me think about something I haven’t seen discussed, which is local collaboratives. The economics makes it seem like a group of people joining together to run good hardware and an open model might make sense, but I haven’t seen anything like this mentioned. Have I been missing it?

I think it would be pretty neat to launch a service helping people who wanted to participate in something like that locate one another.

Re: There is minimal downside to switching to open models

#42

Post by taytus »

CJefferson wrote: post #29
pkulak wrote: post #28

Sure. But OpenAI is the same price. Why would I pay $18/month for z.ai when OpenAI is $20/month?

One big advantage I’ve found — people get attached to models (including me). With open models if you find one that works perfectly for you but the next version doesn’t, you can run the old one forever (or someone will for you)

This is a good point I never thought of. I appreciate it.

Re: There is minimal downside to switching to open models

#43

Post by matheusmoreira »

taytus wrote: post #40
matheusmoreira wrote: post #35

Earlier quoted context omitted.

There's at least the possibility that they intentionally degrade the models as time passes. We can't really verify that we're getting what we're paying for all of the time. All the more reason to invest in local inference.

At current prices, and considering these OS Models' performance, investing in local inference sounds like a bad idea.

Current prices are insane but at this point I'm starting to feel like it's an existential issue. I'm not a US citizen. At any point the USA could come up with some arbitrary export controls. Not having a computer capable of running at least Qwen is starting to actually seem risky to me.

At least it's going to be usable as a very high end gaming PC.

Re: There is minimal downside to switching to open models

#44

Post by blackoil »

bnj wrote: post #41

I’ve been wanting to get better acquainted with local inference but I don’t have the hardware, which has made me think about something I haven’t seen discussed, which is local collaboratives. The economics makes it seem like a group of people joining together to run good hardware and an open model might make sense, but I haven’t seen anything like this mentioned. Have I been missing it? I think it would be pretty nea…

Open models hosted in Cloud???

Re: There is minimal downside to switching to open models

#45

Post by jrm4 »

taytus wrote: post #40
matheusmoreira wrote: post #35

Earlier quoted context omitted.

There's at least the possibility that they intentionally degrade the models as time passes. We can't really verify that we're getting what we're paying for all of the time. All the more reason to invest in local inference.

At current prices, and considering these OS Models' performance, investing in local inference sounds like a bad idea.

At current "proprietary inference company behavior," investing in local inference sounds like the exceedingly far more rational option.

Long term predictability ought to far outweigh a few more cycles of performance.

Re: There is minimal downside to switching to open models

#46

Post by blindriver »

As someone that has pretty powerful desktop that I've been using with local open weight models, people are far exaggerating the quality of them. Some of them are now useful. They don't compare yet to the online models of ChatGPT, Claude, Gemini, etc. They are still about 18 months behind. I have accomplished useful work with them, like image classification on Gemma4, but they are much much slower, much much more expensive and they don't scale at all.

A $10,000 RTX 6000 Blackwell card will pay for 500 months of Claude or Codex, which is 40 years worth of compute. Obviously they are going to raise their prices, my prediction being to $200-500/month, but that still makes them at least years of compute and they scale very well with more traffic. Single GPUs do not, they are pegged at 100% and good luck getting it to answer multiple queries at the same time.

Re: There is minimal downside to switching to open models

#47

Post by itwaswatson »

julianlam wrote: post #6

I think it's interesting that people write off open weight models because they're "a few months behind" proprietary models. I know LLMs move at the speed of light (especially these past few quarters), but if Opus and GPT "a few months ago" were really like open weight models, then there's really no reason to not switch, especially for those who were using these models a few months ago. Your codebase didn't change, so…

We have a provider with Deepseek V4 flash at our work. It can handle 95% of the "actually functional" workload at a tenth of the cost. I still pull up beefier ones sometimes, but that's after some consideration.

The moat is so flat, it only gives +1 food and +1 production. +1 gold with a road.

Re: There is minimal downside to switching to open models

#48

Post by itake »

CJefferson wrote: post #29
pkulak wrote: post #28

Sure. But OpenAI is the same price. Why would I pay $18/month for z.ai when OpenAI is $20/month?

One big advantage I’ve found — people get attached to models (including me). With open models if you find one that works perfectly for you but the next version doesn’t, you can run the old one forever (or someone will for you)

But… the models will fall behind. As libraries and languages and tool calling updates or the world knowledge changes, the models decay.

Personally, I don’t like the change, but it’s just how technology works so I’d rather move with the flow than try to stick my foot down and freeze time.

Re: There is minimal downside to switching to open models

#49

Post by markerz »

bnj wrote: post #41

I’ve been wanting to get better acquainted with local inference but I don’t have the hardware, which has made me think about something I haven’t seen discussed, which is local collaboratives. The economics makes it seem like a group of people joining together to run good hardware and an open model might make sense, but I haven’t seen anything like this mentioned. Have I been missing it? I think it would be pretty nea…

There are plenty of providers of open models that offer very affordable rates. Generally, I recommend looking at OpenRouter since they track various metrics for the various providers.

Re: There is minimal downside to switching to open models

#50

Post by bonesss »

realusername wrote: post #39
kgeist wrote: post #11

Earlier quoted context omitted.

Every new proprietary model is "groundbreaking" and "look, it just solved task X that no other model could solve," only to be referred to as "that crappy previous-generation model" a month later. So yeah, I'm totally fine using Kimi-2.7, GLM-5.2 or Deepseek-v4. I think we've already hit the ceiling and most improvements now seem to be from harness improvements and slightly better RL to improve reasoning/tool calling.

There's also a lot of benchmark trickery going on, it's becoming harder to see how the latest models really improved. The top models also seem to have inconsistent performance depending on the time of day and how far we are from the next release.

I’m an LLM fan, but from an engineering perspective the idea of building atop services that palpably fluctuate in capacity, performance, and capability is nutty.

Even with minor automation I feel like I can watch OpenAI and Anthropic engineers fiddling in real-time. Tuesdays behaviour changes by Thursday, 10AMs production isn’t possible at 11:30AM. Nutty.

Post reply on HN