Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The gap between open weights LLMs and closed source LLMs

blog.doubleword.ai

Post reply on HN
Print view View original

91–100 of 264 posts

Re: The gap between open weights LLMs and closed source LLMs

#91

Post by Bolwin »

NitpickLawyer wrote: post #9
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

> Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever.

In theory yes, but the average person can't really run the big open models.

This is already happening, try to find a provider that still hosts older, especially less popular or succeeded open models.

For me personally, I've been trying to access Kimi K2-0711. There seems to be only one provider left on openrouter (NovitaAI) and 3/4 requests error out

Re: The gap between open weights LLMs and closed source LLMs

#92

Post by bijowo1676 »

Octoth0rpe wrote: post #70
elisbce wrote: post #49

Earlier quoted context omitted.

Chinese frontier models don't need to catch up in every category. They just need to win in coding and that's exactly where they are going. The gap went from 12+ months to 1-2 months with the latest release of GLM 5.2 and coding is a task that you don't need heroic efforts to find rare and long-tail training data, you can just outsmart your competitor by optimizing algorithms and training recipes. This is something th…

> They just need to win in coding and that's exactly where they are going. They don't even need to 'win' in the sense of maxing the benchmark. They can be 20% worse/50% cheaper and many of us (and our managers who approve our token budgets) will be in. Deepseek is 30x cheaper for input/75x cheaper for output than sonnet on openrouter, and it's not a whole lot worse for many things.

Anthropic/OpenAI's valuations are built on assumption of capturing most of the market and having the pricing power to jack up prices for tokens.

It is enough to kneecap their pricing power to trigger the valuation reset by an order of magnitude and humble them a bit.

Plus there are always infrastructure and hardware providers who want to keep their share of profits and will squeeze Anthropic's margins to deflate their valuation (nvidia, aws, RAM manufacturers, etc)

Re: The gap between open weights LLMs and closed source LLMs

#93

Post by g023 »

fridder wrote: post #11
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

We need a SETI@Home but for model training

Slap the gpus in a car and offset the cost of ownership by supplying the grid for GPU power on the go. Either get paid in rebates or tokens. Contribute to a distributed training/inferencing network.

Re: The gap between open weights LLMs and closed source LLMs

#95

Post by anticorporate »

echoangle wrote: post #82
anticorporate wrote: post #44

Earlier quoted context omitted.

There's also, importantly, a distinction between what are told we can no longer use, and what can actually be taken away. Open source and open hardware can be called illegal by a government, but, if we collectively invest our energy into open alternatives, they can't be taken away in the same sense. I can build a RepRap printer and I can use a local AI model. It's on all of us to make sure that the open alternatives…

Believe me, if the government wants to stop you from having access to something like that, they could do it. Just give people some incentive to report you and make really harsh punishments and everyone will be thinking really hard about how bad they want have access.

Well, sure. The same could be said of any freedom they want to take away. The responsibility is on us to preserve those freedoms. Free software, open hardware, right to repair, privacy tools, etc. will all be the weapons of the people in the fight against totalitarianism.

Re: The gap between open weights LLMs and closed source LLMs

#96

Post by eunos »

doctoboggan wrote: post #59

If the Chinese government is as involved in LLM development strategy as many people claim, wouldn't you expect them to immediately cease releasing open weight models and restrict access as soon as they start producing the frontier models? I am assuming this is what the USG thinks and is why they are trying to cut off the flow to foreign nationals ASAP. LLMs are an undeniably valuable tool, and governments like to con…

Xi Jinping isnt as AGI pilled as US govt. CapEx in US is significantly focused on AI related things like chips and data center. It's more diversified in China as they also invests hugely on renewables, EV, BESS, etc.

Re: The gap between open weights LLMs and closed source LLMs

#97

Post by veqq »

Bolwin wrote: post #91
NitpickLawyer wrote: post #9

Earlier quoted context omitted.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

> Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. In theory yes, but the average person can't really run the big open models. This is already happening, try to find a provider that still hosts older, especially less popular or succeeded open models. For me personally, I've…

> NovitaAI is a low cost provider who's strategy seems to be to host as many models as possible for the lowest cost possible so that OpenRouter's routing algorithm will default to them as often as possible. The problem is that they clearly don't spend much time on actually testing and configuring all of the models they provide. There's a reason they are very often the first provider to host a new model. I also suspect that they run models at lower quants than they claim but that is not something I can prove. https://www.reddit.com/r/LocalLLaMA/comments/1mk4kt0/be_care...

Re: The gap between open weights LLMs and closed source LLMs

#98

Post by verdverm »

sdesol wrote: post #73
doctoboggan wrote: post #59

If the Chinese government is as involved in LLM development strategy as many people claim, wouldn't you expect them to immediately cease releasing open weight models and restrict access as soon as they start producing the frontier models? I am assuming this is what the USG thinks and is why they are trying to cut off the flow to foreign nationals ASAP. LLMs are an undeniably valuable tool, and governments like to con…

I talked about this before but China would be in much better position if LLMs turn into a commodidty. Where they can dominate is in hardware, as fast and cheap inference is probably going to be the moat.

My futurology is that most of us will end up on unlimited token plans like we are for mobile data. We don't need the very best model for most tasks and the trend in computing has always been towards cheaper and more efficient unit economics. I do not see this ending any time soon.

Re: The gap between open weights LLMs and closed source LLMs

#99

Post by verdverm »

Gigachad wrote: post #54
llmslave wrote: post #16

The gap is huge and im tired of reading these articles constantly

Are you talking about hosted vs the ones you can easily run locally? Because there are open models that require hundreds of gb of vram which are apparently pretty close.

on the Will It Mythos benchmark, small models are punching way above their weight(s)

gemma4-26B (#7)

qwen-3.6-27B (#9)

https://news.ycombinator.com/item?id=48640196

Re: The gap between open weights LLMs and closed source LLMs

#100

Post by Gigachad »

verdverm wrote: post #99
Gigachad wrote: post #54

Earlier quoted context omitted.

Are you talking about hosted vs the ones you can easily run locally? Because there are open models that require hundreds of gb of vram which are apparently pretty close.

on the Will It Mythos benchmark, small models are punching way above their weight(s) gemma4-26B (#7) qwen-3.6-27B (#9) https://news.ycombinator.com/item?id=48640196

I've tried running qwen 3.6 locally and it felt like LLMs a year ago where you can get them to do some stuff but the tasks have to be very small and you have to course correct them a lot to the point it's hard to say it's any faster than doing it all yourself.

Certainly the gap is closing but I feel it still makes more sense to pay pennies to run the full sized open models hosted on much better hardware.

Post reply on HN