Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The gap between open weights LLMs and closed source LLMs

blog.doubleword.ai

Post reply on HN
Print view View original

131–140 of 264 posts

Re: The gap between open weights LLMs and closed source LLMs

#131

Post by ggm »

knowaveragejoe wrote: post #114
ggm wrote: post #80

Earlier quoted context omitted.

Why do you think this matters? Not that it does or doesn't but what quality does "US WINS" or "CHINA WINS" bring to the table?

I think the unspoken fear is that if we assume one or the other will "win" in reaching AGI(or whatever threshold of capability), the rest of the world will sooner or later live under their system of rule as a consequence

I very much doubt the primary reason nation states are lining up to permit or forbid access to these systems is 'fear of future AGI dominance'

I think it's much more immediate/present: the weights and the information breach significant strategic controls on national data and posture, which can be back-derived from the models. If you can analyse a model, you can infer what structural inputs dictate it.

Re: The gap between open weights LLMs and closed source LLMs

#132

Post by NemoNobody »

dvngnt_ wrote: post #84
echoangle wrote: post #82

Earlier quoted context omitted.

Believe me, if the government wants to stop you from having access to something like that, they could do it. Just give people some incentive to report you and make really harsh punishments and everyone will be thinking really hard about how bad they want have access.

They can stop piracy or child predators. what makes you think they can prevent access to running models that require no internet access to run

Piracy is in a practical Golden Age rn and the Epstein Files exist - so the Government doesn't really do either of those things very well at all.

Plus for a certain type of person "Piracy" is more of a philosophical belief or political position - there are fundamentalist equivalent, very proficient, "Pirates" who will under no circumstances stop and are not doing it for money. There are obviously an enormous amount who are in it for the money - "big brand names" now reportedly comprise as high as 63% of the advertising on illicit piracy sites - I'm too lazy to get the link, that sentence ought to be enough tho if you want to look into that bizarre reality.

I'm not certain either of those things are in the Government's direct control - both require society at large to share the belief and essentially choose not to do said activities.

(Regarding your second example, unfortunately most abusers are people children know, the Epstein Class was supposed to be just Q Anon crazy conspiracy stuff, none of this is ok in any fashion. Both exist, one local entirely beyond the government - the other appears to have incorporated people from government.)

My point is simply this - WE determine what the Government can do. What we believe matters more than anything else. Don't ever discredit The People's ability - we are pretty awesome.

Re: The gap between open weights LLMs and closed source LLMs

#133

Post by alfiedotwtf »

profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

Exactly my worry. I’m optimistic in the future the EU, the EFF, the GNU, or the Linux Foundation could have been the umbrella to run a LARGE open model for everyone.

It’s sad to think that Mozilla spent years and millions doing virtual reality and AI, they would have been perfect to do this but let’s face it - who knows if Mozilla will be around even 5 years from now

Re: The gap between open weights LLMs and closed source LLMs

#134

Post by alfiedotwtf »

NitpickLawyer wrote: post #9
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

Yeah, but the biggest plus for open models is that they can never be taken away. In other words, whatever capabilities they reach (even if there will never be another model), those stay forever. That can't be said for API-based models where a provider can sunset models whenever they feel like (i.e. gpt5-mini will soon be gone, and replaced by a more expensive 5.4-mini, same for goog, etc). And there will always be in…

> And the chinese labs also have incentives to keep releasing models

Not really.

Re: The gap between open weights LLMs and closed source LLMs

#135

Post by alfiedotwtf »

christina97 wrote: post #35

The Chinese models will not overtake the frontier US ones given the current way things are going. The US models derive their lead from incredible efforts to source more and higher quality (mostly synthetic data) via great feats (eg generating with humongous teacher models that could never feasibly serve interactive traffic). The Chinese models advance via heroic efforts to optimize models and great feats to secure mo…

> source more and higher quality (mostly synthetic data)

Kind of an oxymoron don’t you think.

If they could generate data that looked kind of real, why don’t they just generate that data on the fly during inference

Re: The gap between open weights LLMs and closed source LLMs

#136

Post by Eridrus »

profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

I think the bigger issue is the ever increasing capital requirements, which may cause even the closed weight companies to fall away from the frontier, e.g. Google & Meta are barely hanging on. For Google it feels a bit existential to remain at the frontier, but even then they're barely there.

I hope that we find ways of continuing to improve these models besides continuing to exponentially increase capex spend until all but one of your competitors falls away.

Re: The gap between open weights LLMs and closed source LLMs

#137

Post by Onavo »

Eridrus wrote: post #136
profsummergig wrote: post #5

IMHO, the biggest problem with the future of open weights models is that currently, open weights models are the result of philanthropy by some private org. (e.g. DeepSeek). The spigot can be turned off at any time. Until there's some sort of "community owned hardware", open weights models are always at risk of being discontinued.

I think the bigger issue is the ever increasing capital requirements, which may cause even the closed weight companies to fall away from the frontier, e.g. Google & Meta are barely hanging on. For Google it feels a bit existential to remain at the frontier, but even then they're barely there. I hope that we find ways of continuing to improve these models besides continuing to exponentially increase capex spend until…

Google and Meta's failures are more due to mismanagement no?

Re: The gap between open weights LLMs and closed source LLMs

#139

Post by FooBarWidget »

yorwba wrote: post #83
throwawayffffas wrote: post #74

Earlier quoted context omitted.

I don't think that's the case, it's not philanthropy, they are getting something out of it. The labs are learning from one another from the shared models. Plus I am certain it makes financial sense. I am guessing here but fully utilizing a subscriptions limits probably costs the operator more money than the subscription revenue, that is why anthropic is making such a big stink about the chinese data harvesting. By re…

The primary benefit of releasing weights is the attention it generates. Some people have the hardware to run it, try it out because it's free, tell everyone about it, and then even people who don't have the hardware might get interested and pay the original developer. So it's a marketing expense, basically. The most popular LLM product in China is Bytedance's Doubao. You probably haven't heard of them since they neve…

That's not meaningfully different from philanthropy. If Chinese AI products generate sufficient revenue with cheaper marketing strategies, then the incentives for releasing open models will go away.

Right now, there is a shortage of talented researchers, and the attention that open models generate allow them to attract good hires. But this is a fragile dynamic that can break in the future. It's not very different from commercial open source work, except it's much more capital intensive and lower volume.

Re: The gap between open weights LLMs and closed source LLMs

#140

Post by stkdump »

jstanley wrote: post #118
cedws wrote: post #81

I haven’t seen it discussed anywhere that closed models can essentially cheat benchmarks right? What Anthropic or OpenAI brand as a model doesn’t necessarily have to be just weights, it can be a whole backend system that augments the model itself. With this they can score better benchmarks than an open source model that is weights alone.

Sure, I think that's fine, that all counts. It counts for open source too, it's not like they're somehow running these benchmarks without any harness. Nobody cares if your AGI is 100% made out of neural networks or if it's like 50% neural networks and 50% perl scripts.

I think they mean cheat in a Dieselgate sense. You detect that you are being tested with a specific benchmark question and heuristically give the correct (manually programmed) answer. That wouldn't be AGI.
Post reply on HN