Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The gap between open weights LLMs and closed source LLMs

blog.doubleword.ai

Post reply on HN
Print view View original

21–30 of 264 posts

Re: The gap between open weights LLMs and closed source LLMs

#21

Post by notnullorvoid »

profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

> Until there's some sort of "community owned hardware"

Or until some bright people figure out drastically more efficient means of training.

Re: The gap between open weights LLMs and closed source LLMs

#22

Post by redox99 »

felooboolooomba wrote: post #18
NitpickLawyer wrote: post #9

Earlier quoted context omitted.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

> they can never be taken away Your right to 3d print whatever you want is about to be taken away (in California). What software you can run on your computer can already be restricted. Absolutely everything can be taken away. The simplest way to remove open models is probably to declare them a tool that terrorists could use. Crazy? Yes, the world is totally crazy these days.

That only affects people in California. Whereas Fable being shut down affects people all over the world.

Re: The gap between open weights LLMs and closed source LLMs

#23

Post by komadori »

throwuxiytayq wrote: post #10
NitpickLawyer wrote: post #4

Earlier quoted context omitted.

Literally no one cares. There are "full" open certified GMO free grass fed training data blah blah models. Apertus, Olmo, etc. No one cares. For all intents and purposes people use the term to describe a model that you can run locally and are allowed to modify and re-release. The rest is useless semantics. No one can "rEpRoDuCe" a model anyway.

No-one cares to quit social media or stop using Windows, but it’s a goal worthy of discussion all the same. The name is bad, doesn’t even make any fucking sense and it gives open source a bad rep.

I wouldn't say that no one cares, but obviously many fewer people care when the cost of "recompiling" a model from its open source training pipeline is so high. Also, if you only have the weights, you can still use it to generate training data for a new model (i.e. distillation) so it's inherently less locked down then closed source binaries were.

Re: The gap between open weights LLMs and closed source LLMs

#24

Post by amluto »

jacobgold wrote: post #6

It would be interesting to know how much of a boost the closed models companies are giving the open models. If the closed models stop improving will the progress of open models slow?

> It would be interesting to know how much of the "distillation" boost is helping the open weight models keep up.

Some people in China surely know.

> Like if the closed models stop improving will all the closed models also stop improving?

Seems extremely unlikely, unless the models all hit some kind of wall soon. The Chinese companies may be behind the US in compute capacity, but they have excellent researchers [0] who are probably approximately as good as their US counterparts at the kind of problem generation and RL that is currently working so well.

I would be very surprised, though, if the models cannot continue to be improved rapidly in any area that allows a tight feedback loop like programming, at least up to the point where we puny humans lose the ability to define objective functions.

(And, conversely, I don’t expect magic in fields where the feedback is slow or expensive. A model is not about to reliably invent a wonderful medicine for the same reason that a large and extremely competent pharma company cannot: the evaluation process is extremely slow and it’s so expensive that the kind of utterly enormous corpus that is driving the current progress in coding is simply not available. Running RL on m iterations of n medication-development trajectories each is going to cost n*m times $10-100 million and take m years if it’s even possible at all.)

[0] The US advantage in this space will likely decline, since the brain drain from the rest of the world via the US university system to US labs is drying up.

Re: The gap between open weights LLMs and closed source LLMs

#25

Post by Azantys »

fridder wrote: post #11
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

We need a SETI@Home but for model training

I think model training is pretty hard to do efficiently on a vastly distributed network. If the model cant fit into the VRAM of the node your performance becomes so bad its useless, so a distributed model could only be properly trained if the size of the model doesnt exceed the majority of the nodes VRAM sizes. Maybe there is a different way of doing training but this would be the only way I can see. And it would still be much worse than just using a big datacenter where everything is fully interconnected. BOINC projects work great because its usually just a lot of small compute and memory required so every old desktop and laptop can contribute. Training a model which can compete and is not tiny requires neither low compute or low memory amount. BOINC tasks take minutes usually or sometimes hours but not weeks or months like training a model from scratch. But something like 7B or lower could maybe be trained like this. Im not sure but I think someone is already working on something like this but I dont remember the name of the project.

Re: The gap between open weights LLMs and closed source LLMs

#26

Post by JumpCrisscross »

Now let’s look at the economics of buying versus renting. I’ve seen a lot of attention given to hardware capital costs. But a comment the other day got me thinking about power costs, too—at what performance differential do these factors intersect to make on-prem economically competitive with datacenters for businesses?

Re: The gap between open weights LLMs and closed source LLMs

#27

Post by amunozo »

jacobgold wrote: post #6

It would be interesting to know how much of a boost the closed models companies are giving the open models. If the closed models stop improving will the progress of open models slow?

Why are we assuming only American labs can innovate? DeepSeek already innovated a lot in efficiency, for example.

Re: The gap between open weights LLMs and closed source LLMs

#29

Post by UncleOxidant »

profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

> The spigot can be turned off at any time.

True. And it's possible that this has already happened at Alibaba Qwen - at least for the smaller models that people had a chance of running at home (122B and smaller).

Post reply on HN