Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-weight AI is having its Kubernetes moment

tobi.knaup.me

Post reply on HN
Print view View original

201–210 of 346 posts

Re: Open-weight AI is having its Kubernetes moment

#201

Post by foobar10000 »

Y_Y wrote: post #181
thih9 wrote: post #16

Is anyone using open weight models for agentic coding? What is your stack (harness, model) and how much do you pay per month? How would you compare your experience to a typical subsidized plan like Claude Code + Pro plan? I’m asking because i keep hearing that open weight models are cheap and efficient - is that really the case in practice?

GLM 5.2 awq4 via Opencode, it was better than corporate's fave Sonnet 4.8 and a but worse than Opus 4.8. overall quite capable of a lot of what the dev team needed. Not as good as Sonnet 5 but also never runs out of tokens. The flip side is that it takes four H200s to run, and that will only let you cache context for maybe three users. Fingers crossed our Blackwells show up and Kimi 3 really releases weights, because…

Glm 5.2 nvfp4 on 4 b300 with dpattn 4 and ram will get you about 20 users live at 400k context - and 60 easily if you give that server 2tb of ram and 4 nvme 8 tb drives. There are some sglang patches needed - but we will be releasing them soon.

Re: Open-weight AI is having its Kubernetes moment

#202

Post by TacticalCoder »

ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

> So, any solution to this “problem” must include ALL open-weight models.

What about the EU? Would they follow Uncle Sam's order to ban all open-weight models? Lately the EU hasn't been that cozy with american companies: there are EU companies and institutions moving to EU clouds, the EU just fine Google a cool billion, several are switching away from Windows to Linux, etc.

Or is it just the US that'd ban open-weights models, while, say, the EU and Japan would still allow them?

Re: Open-weight AI is having its Kubernetes moment

#203

Post by ThomasGlanzmann »

PunchyHamster wrote: post #134
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It would basically make America behind as every other country would use open, cheaper models for all tasks but the ones requiring frontier models. And that list of tasks grows smaller every day > Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart…

I often use deepseek-v4-flash to summarize hn threads for me. Your comment was enough for deepseek to refuse my request: Content Exists Risk.

Re: Open-weight AI is having its Kubernetes moment

#204

Post by andriy_koval »

xprnio wrote: post #199
andriy_koval wrote: post #192

Earlier quoted context omitted.

someone can run tests and see that models output exactly the same results, and then you are open to criminal investigation.

On something that is inherently non-deterministic? Something which is also to a great extent distilled from other frontier models, meaning it has the possibility to generate similar outputs to those meaning that just pattern detection might also not be as effective? Easier to ban everything that’s open, than try to figure out which one of them is Chinese

Now imagine prosecutor found expert, who said there is benchmark which while performing 100k test questions found it is the same model with 98% probability, and then you need under oath testify where did you get this model.

Re: Open-weight AI is having its Kubernetes moment

#205

Post by Danox »

SilverElfin wrote: post #159

What’s interesting is no one is talking about political censorship in models and how DeepSeek, Kimi, and the rest have to abide by CCP rules. It’s a big opportunity for China to control information.

And that censorship will fail too, like sanctions, tariffs, and keeping down open source software or exchanging ideas across borders, ultimately it will fail, but that won’t stop governments across the world from trying.

Re: Open-weight AI is having its Kubernetes moment

#206

Post by andriy_koval »

wyrdcurt wrote: post #198
andriy_koval wrote: post #192

Earlier quoted context omitted.

someone can run tests and see that models output exactly the same results, and then you are open to criminal investigation.

Except that even the exact same model won't output the exact same results, that's a fundamental aspect of how LLMs work. They're probabilistic/stochastic, not deterministic.

Models are weights for matrix operations, they are determenistics.

Re: Open-weight AI is having its Kubernetes moment

#207

Post by spwa4 »

mlyle wrote: post #84
spwa4 wrote: post #17

Earlier quoted context omitted.

Because as per usual it's silicon valley misunderstanding economics. AI is HPC. And how the HPC market worked before: If you're the best performing "computing cluster" (ie. whatever you call the entity that can complete a massive calculation), you get a blank check from Congress. Why? Because you need those calculations to "pump" nuclear weapons. They are needed to calculate both the geometry to make fusion bombs pos…

> Because you need those calculations to "pump" nuclear weapons. They are needed to calculate both the geometry to make fusion bombs possible at all and to calculate the effect of a given geometry. They are the reason US/Russia/China have the biggest and strongest weapons known to humanity. We have a couple new nuclear weapon designs, but not really going for bigger or stronger. Just packaging. We built the big power…

> We have a couple new nuclear weapon designs, but not really going for bigger or stronger. Just packaging.

There's many other considerations. Like the type and amount of fissile material. To name one that became well known: any plutonium needs to be refreshed (re-breeded I believe is the term) every few years.

Also, look at first designs: https://www.bbc.com/news/newsbeat-35242069 The weight is secret, but I think you can easily see it's going to be deep into "extremely impractical" territory.

Surely you can see why someone (especially aircraft designers) might ask for better versions. Ideally you'd like a version that fits on the hypersonic missiles and those things ... are just not going to work. The size. The shape. The weight. None of them will work.

Then a quick theoretical exploration will tell you that the minimum theoretical size of such a device is tiny. The scare was about "suitcase sized", but if you actually do the calculation looking for the minimum ... to do it however you need to create an explosion of the correct shape to get anywhere near those minimum sizes. And explosion simulations are a problem that utterly sucks ... Oh and these are secret military projects, these simulations, not very optimal. The people doing them are best described as loyal, and not as great physicists. Not saying they're terrible, but in the movie Oppenheimer you can clearly see why the best and brightest are not available for these things.

Re: Open-weight AI is having its Kubernetes moment

#208

Post by realusername »

TacticalCoder wrote: post #202
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

> So, any solution to this “problem” must include ALL open-weight models. What about the EU? Would they follow Uncle Sam's order to ban all open-weight models? Lately the EU hasn't been that cozy with american companies: there are EU companies and institutions moving to EU clouds, the EU just fine Google a cool billion, several are switching away from Windows to Linux, etc. Or is it just the US that'd ban open-weight…

I doubt Anthropic and OpenAI have enough weight to also make the EU ban open models, it would remain a US thing.

Re: Open-weight AI is having its Kubernetes moment

#209

Post by Danox »

kalu wrote: post #56

The sentiment in this article is nice. But open source software is a weak analogy for frontier models. Principally because software requires zero capital investment (actually zero) while frontier models demand billions. Open models can only survive in the long run if they can (eventually) generate significant cash flows or if they are paid for by governments. Now China essentially has a monopoly on open weight models…

They don’t have a monopoly and if they do, it would only be because the EU and the United States let them or I should say they let all their decisions be made by private companies who scraped the public Internet and now want to fence it in for their up-and-coming IPO.

Re: Open-weight AI is having its Kubernetes moment

#210

Post by andsoitis »

DennisP wrote: post #129
andsoitis wrote: post #102

Earlier quoted context omitted.

So?

So pharmaceutical companies spend far less on marketing outside the US, partly because every other country besides New Zealand makes those incessant drug ads illegal, and partly because governments negotiate prices and keep profit margins down. If the argument is that R&D costs are what make drugs expensive, then we could easily eliminate an even greater expense by just copying what other developed nations do.

Don’t you think drug companies would be doing that (not advertise) if it would increase their profit?
Post reply on HN