Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The gap between open weights LLMs and closed source LLMs

blog.doubleword.ai

Post reply on HN
Print view View original

31–40 of 264 posts

Re: The gap between open weights LLMs and closed source LLMs

#31

Post by jfim »

NitpickLawyer wrote: post #9
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

True, but the capabilities and knowledge of that model are also frozen in time, so the value of that model declines over time.

A model that writes code without knowledge of any language or library changes for half a decade is less useful. A 2021 era chatgpt would be quite quaint in 2026.

Right now the Chinese labs might have incentives to release their models for free, and maybe Google is happy to release open weights today, but I'm sure there are already bean counters at Google salivating at the idea of having Gemini in Chrome as part of a Google AI monthly subscription just like YouTube premium and other Google subscriptions.

Re: The gap between open weights LLMs and closed source LLMs

#32

Post by UncleOxidant »

NitpickLawyer wrote: post #9
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

> Nvda for one has every incentive to keep the nemotron line going

They're releases so far have been kind of lackluster compared to Qwen and other Chinese models. My suspicion is that Nvidia won't be releasing models that appear to compete with frontier models because that would upset their big customers.

Re: The gap between open weights LLMs and closed source LLMs

#33

Post by dabinat »

I believe the open model party will eventually end. Perhaps because companies realize it’s too much of a commercial advantage, countries don’t want to give other countries commercial or military help, or maybe even an outright ban after someone uses an open model to guide them through how to make a bomb.

Re: The gap between open weights LLMs and closed source LLMs

#34

Post by _pdp_ »

Frankly it does not matter if there is gap because for most practical use-cases the end user can barely perceive the difference in intelligence.

On paper frontier models will be ahead of the curve but I don't think hardly anyone will be able to tell if a piece of work, say a landing page, is created with Fable or GLM and that is the point. The perceptible intelligence will reach a point beyond which it is no longer considered, except for some narrow use-case.

Re: The gap between open weights LLMs and closed source LLMs

#35

Post by christina97 »

The Chinese models will not overtake the frontier US ones given the current way things are going. The US models derive their lead from incredible efforts to source more and higher quality (mostly synthetic data) via great feats (eg generating with humongous teacher models that could never feasibly serve interactive traffic). The Chinese models advance via heroic efforts to optimize models and great feats to secure more and higher quality training data from the US frontier models.

For an (Chinese) open weight model to surpass the (US lab) frontier models, this equation must flip and the Chinese labs must entirely retool from harvesting frontier model data to producing the data systems and efforts to produce novel data; as well as procuring latest generation hardware en masse for this. This does not happen easily. Also training a frontier scale model is actually not such an unimaginable feat: doing all the inference with the teacher models is where the hardware goes.

Re: The gap between open weights LLMs and closed source LLMs

#36

Post by baby_souffle »

0x3f wrote: post #17
fridder wrote: post #11

Earlier quoted context omitted.

We need a SETI@Home but for model training

Consumer hardware over the internet is not really suitable for this, AFAIK.

There's some really early days work on making training loops robust to failure but they all have trade-offs right now.

I remain hopeful that we'll be able to democratize the entire tech stack for this tech.

Re: The gap between open weights LLMs and closed source LLMs

#37

Post by ainka-ainka »

fridder wrote: post #11
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

We need a SETI@Home but for model training

Here's a project trying that - https://nousresearch.com/nous-psyche

Re: The gap between open weights LLMs and closed source LLMs

#38

Post by judge2020 »

NitpickLawyer wrote: post #4
samat wrote: post #2

Article confuses open source models with open weights models. Not the same thing. It’s used right in the articles body, but title is misleading.

Literally no one cares. There are "full" open certified GMO free grass fed training data blah blah models. Apertus, Olmo, etc. No one cares. For all intents and purposes people use the term to describe a model that you can run locally and are allowed to modify and re-release. The rest is useless semantics. No one can "rEpRoDuCe" a model anyway.

open source vs source-available. Companies taking an extremely cautious approach to AI can't use source data that is potentially a violation of copyright (pending worldwide court decisions and/or regulation on said topic). Although that cat is already out of the bag for basically every stock-traded company using LLMs trained on non-licensed data, so I don't see there being much actual risk in using them.

Re: The gap between open weights LLMs and closed source LLMs

#39

Post by typs »

amluto wrote: post #24
jacobgold wrote: post #6

It would be interesting to know how much of a boost the closed models companies are giving the open models. If the closed models stop improving will the progress of open models slow?

> It would be interesting to know how much of the "distillation" boost is helping the open weight models keep up. Some people in China surely know. > Like if the closed models stop improving will all the closed models also stop improving? Seems extremely unlikely, unless the models all hit some kind of wall soon. The Chinese companies may be behind the US in compute capacity, but they have excellent researchers [0] w…

Perhaps. RL env companies based in the U.S. sell to Chinese labs quite a bit too though (though on a discount, once they're no longer on the frontier)! And it would make sense that a lot of these problems which are based on work in the U.S. enterprise economy would be coming from the U.S.

Re: The gap between open weights LLMs and closed source LLMs

#40

Post by vitally3643 »

felooboolooomba wrote: post #18
NitpickLawyer wrote: post #9

Earlier quoted context omitted.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

> they can never be taken away Your right to 3d print whatever you want is about to be taken away (in California). What software you can run on your computer can already be restricted. Absolutely everything can be taken away. The simplest way to remove open models is probably to declare them a tool that terrorists could use. Crazy? Yes, the world is totally crazy these days.

Just like declaring piracy illegal stopped piracy and removed pirated materials from everyone's computers.

Everything cannot, in fact, be taken away. Don't propagandize yourself. Some things, like information, are free. Not even China can prevent all its citizens from accessing Western internet. USGov simply does not have the resources to find and audit every hard drive and USB stick in the country for illegal files. The internet cannot be censored 100% without literally cutting every cable and confiscating every radio.

The software that runs on my computer cannot, in fact, be restricted. It can be declared illegal, but there literally is no mechanism by which it can be enforced other than a government goon standing over my shoulder 24/7.

Some freedoms really cannot be removed without utterly implausible amounts of effort. Arguing otherwise is helping to erode freedom. So stop it.

Post reply on HN