The gap between open weights LLMs and closed source LLMs
blog.doubleword.ai
The gap between open weights LLMs and closed source LLMs
1–10 of 264 posts
Re: The gap between open weights LLMs and closed source LLMs
#2Article confuses open source models with open weights models.
Not the same thing.
It’s used right in the articles body, but title is misleading.
Re: The gap between open weights LLMs and closed source LLMs
#3Achilles and the tortoise [0] is usually a fallacy. If the tortoise has a head start, then Achilles will never catch it because in the time it takes Achilles to reach the tortoise's location the tortoise has moved some degree further, ad infinitum. Obviously not real because Achilles will pass the tortoise -- I think a fallacy because the framing creates a fake asymptote (they will both pass the point where they're approaching a tie).
In this case it may actually apply though, no? Open models get better from closed model distillation?
Re: The gap between open weights LLMs and closed source LLMs
#4Article confuses open source models with open weights models. Not the same thing. It’s used right in the articles body, but title is misleading.
Literally no one cares. There are "full" open certified GMO free grass fed training data blah blah models. Apertus, Olmo, etc. No one cares. For all intents and purposes people use the term to describe a model that you can run locally and are allowed to modify and re-release. The rest is useless semantics. No one can "rEpRoDuCe" a model anyway.
Re: The gap between open weights LLMs and closed source LLMs
#5IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek).
The spigot can be turned off at any time.
Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.
Re: The gap between open weights LLMs and closed source LLMs
#6It would be interesting to know how much of a boost the closed models companies are giving the open models.
If the closed models stop improving will the progress of open models slow?
Re: The gap between open weights LLMs and closed source LLMs
#7IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.
It's just a smart business decision that allows their models to compete and gain market-share against much pricier private models. No philanthropy there.
Re: The gap between open weights LLMs and closed source LLMs
#8Article confuses open source models with open weights models. Not the same thing. It’s used right in the articles body, but title is misleading.
I was advocating for "available weight" as a value neutral term for a while.
I gave up. No one cares. And no one will ever tell the truth about the training anyways.
Substantial and growing freedom beats zero freedom ever again.
Re: The gap between open weights LLMs and closed source LLMs
#9IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.
Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc).
And there will always be incentivised parties that release models. Nvda for one has every incentive to keep the nemotron line going, as they're directly profiting from people running this. And the models aren't really far from open SotA anyway.
Goog will probably continue to release the small models, since they'll use them for browser stuff anyway, and know that they'll leak. So for them it's a win-win to release the small models and gain some dev market share.
And the chinese labs also have incentives to keep releasing models, and will likely continue to get gov support to do so (yay commercial wars between nations).
Re: The gap between open weights LLMs and closed source LLMs
#10Article confuses open source models with open weights models. Not the same thing. It’s used right in the articles body, but title is misleading.
Literally no one cares. There are "full" open certified GMO free grass fed training data blah blah models. Apertus, Olmo, etc. No one cares. For all intents and purposes people use the term to describe a model that you can run locally and are allowed to modify and re-release. The rest is useless semantics. No one can "rEpRoDuCe" a model anyway.
No-one cares to quit social media or stop using Windows, but it’s a goal worthy of discussion all the same.
The name is bad, doesn’t even make any fucking sense and it gives open source a bad rep.