Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-weight AI is having its Kubernetes moment

tobi.knaup.me

Post reply on HN
Print view View original

241–250 of 346 posts

Re: Open-weight AI is having its Kubernetes moment

#241

Post by michaellee8 »

bfung wrote: post #165
ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It’s not really feasible, in my opinion. US Gov could make US companies comply, like have Huggingface take down models out of compliance. But most likely, a foreign-to-US Huggingface replacement would be made and everyone would go there instead. Lose-lose for US.

It is called ModelScope

Re: Open-weight AI is having its Kubernetes moment

#242

Post by wredcoll »

octopoc wrote: post #100
ncallaway wrote: post #38

Earlier quoted context omitted.

The government backed monopoly to ensure that supply remains artificially restricted to ensure that the market will support the higher prices is

Without temporary monopolies granted by patents, those prescription medications wouldn’t exist in the first place.

What percentage of research is performed outside of the companies? (by college/government)

Re: Open-weight AI is having its Kubernetes moment

#243

Post by andriy_koval »

m11a wrote: post #226
andriy_koval wrote: post #219

Earlier quoted context omitted.

That French guy takes risk to be forever under US warrants for breaking American law, denied access to financial institutions even in Europe and will quickly go to some KYC entity list, and you will be notified as his clients to stop using his model. Or you think all kind of fraud can be committed through some "french guy"? Also, I am not confident, receiving illegal materials from French guy gates you from personal…

I presume such US legislation isn't going to try claim worldwide jurisdiction to block all persons worldwide from using Chinese models. In which case, the French guy wouldn't be violating American law. As for the American company, it's pretty difficult to check the provedance of open weights. It's even difficult to check the provedance of open source code, because chains of attribution aren't always clear. I posted e…

> I presume such US legislation isn't going to try claim worldwide jurisdiction to block all persons worldwide from using Chinese models. In which case, the French guy wouldn't be violating American law.

legislation will block importing Chinese models to the US

> As for the American company, it's pretty difficult to check the provedance of open weights.

government or some companies can build benchmarks/system which will give y/n answer

Re: Open-weight AI is having its Kubernetes moment

#244

Post by nunez »

ozgung wrote: post #126

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive at…

It'll be pretty easy. Some US gov entity will create a list of models from Hugging Face, decree "thou shalt not provide access to these models", wrap it around some scary legalese for the pirates who try and that will be that.

Though the legalese might not even be necessary. The list alone will make sure that no American company runs these on their servers, including the hosting providers.

Re: Open-weight AI is having its Kubernetes moment

#245

Post by AnduCrandu »

theshrike79 wrote: post #221
xdennis wrote: post #212

Earlier quoted context omitted.

How is that in any way equivalent? The American government/legislature doesn't force you to adopt any particular view of the 2020 election. You can say Biden won or you can say it was defrauded by dead people and Trump actually won. You're allowed to say either one and you can train an LLM to say either one.

...have you seen the current administration being grilled by congress members? They are actually properly unable to say "Biden won the 2020 election" because that's against Dear Leader's views. They will say anything similar like "was confirmed as...", but those exact words will NEVER come out of their mouth.

Yes, but do you have any indication that this applies to American LLMs?

Re: Open-weight AI is having its Kubernetes moment

#246

Post by Krei-se »

munchler wrote: post #172
what-is-water wrote: post #133

Earlier quoted context omitted.

Kubernetes orchestrates your container workloads over a cluster, which consists of virtual/bare metal machines(nodes). This means you can tell the kubernetes API "I want to run a container workload" and it will be started on one of the nodes that form the cluster, unlike e.g. Docker, where a docker daemon belongs to a specific node. If you remove the node your workload is running on from the cluster the workload will…

Thank you for this explanation. It makes sense, but I don't really understand why it has become so popular. Professionally, my experience is that certain software components need to run together on an individual machine (e.g. database server, app server, web server), and then those machines need to be networked in a certain way (e.g. web server talks to app server, which talks to database server), so I really need to…

Your host with the webserver has no idea its just a docker image. It talks to app.domain.tld and gets a reply. app.domain.tld asks db.domain.tld with an SQL query and gets a reply. But it can be one of any deployed docker image on any bare metal host - which one is db. and app. etc. is decided by Kubernetes.

In the background kubernetes routes all these docker images with each other without you having to think about this. You may have 1000 db.domain.tld nodes caching a master db host - if you configure that in the docker image, that's not different to bare metal replication.

Same with load balancing in webservers. You can do that! Or you can just have kubernetes handle it. I'm not sure but would expect it to swap images in a way network is optimized - i don't use this kind of software. I just know it's done like this because the bottleneck of modern software is not the local network so its not an issue to have these pieces on different bare metal hosts. And its easier to stay operative if some hosts fail - but you can all solve this by hand.

I work with bottlenecks between network, ssd, ram and vram but if you deploy npm riddled software for >1 Million users well then you may want Kubernetes. Or if you are google and have 10k engineers that need to agree on a standard!

If you can do this yourself with load balancing, replication etc - do it yourself and don't think there's something wrong with that. It's more elegant, efficient - but you have to agree with others how you do it. And that can also be a bottleneck ;)

Still i think you doing it by hand is better so don't worry you are not old you may just have higher standards.

Re: Open-weight AI is having its Kubernetes moment

#247

Post by theshrike79 »

AnduCrandu wrote: post #245
theshrike79 wrote: post #221

Earlier quoted context omitted.

...have you seen the current administration being grilled by congress members? They are actually properly unable to say "Biden won the 2020 election" because that's against Dear Leader's views. They will say anything similar like "was confirmed as...", but those exact words will NEVER come out of their mouth.

Yes, but do you have any indication that this applies to American LLMs?

Not yet, but we're only 1,5 years into this current regime.

They've already hit Anthropic once with a big government hammer (Fable release). There's no indication they won't do it again to force the LLM to have Correct Opinions.

Re: Open-weight AI is having its Kubernetes moment

#248

Post by slim »

Foobar8568 wrote: post #94
nyrikki wrote: post #50

Earlier quoted context omitted.

I don’t know if others would find this useful, but previous did have custom harnesses etc.. but tools have improved so much that I drastically simplified. That said, even the foundational models fail at the hard parts of my code so I use it opportunistically. I have reduced down to just using zed, will three locally hosted models. Qwen 3.6 27b on 1x3090 llama.cpp with 128k context ~50tps Qwen 3.6 35B-A3B on 1x titan…

You wouldn't get 100 tps on a Qwen 3.6 35b with a 5060 (or two) when a 5090 can barely reach that.

I get 66 tps on one. so 100 on two seems plausible to me

Re: Open-weight AI is having its Kubernetes moment

#249

Post by nunez »

applicative wrote: post #170

Everyone just keeps assuming, as if it were the law of gravity, that China will continue in perpetuity to deliver the weights of its 'frontier' models to Hugging Face. Its Mythos moment is a few months away and there is plenty of reporting suggesting their response will be similar, which is anyway obvious. It baffles me that anyone can seriously believe that China is going to put its Mythos successor on Hugging Face…

They very well might. Read up on the infrastructure projects China has provifed to Africa in exchange for oil refineries and drilling sites across the continent: https://oilprice.com/Energy/Crude-Oil/China-Is-Rapidly-Expan.... Roads, schools and more.

Re: Open-weight AI is having its Kubernetes moment

#250

Post by applicative »

nunez wrote: post #249
applicative wrote: post #170

Everyone just keeps assuming, as if it were the law of gravity, that China will continue in perpetuity to deliver the weights of its 'frontier' models to Hugging Face. Its Mythos moment is a few months away and there is plenty of reporting suggesting their response will be similar, which is anyway obvious. It baffles me that anyone can seriously believe that China is going to put its Mythos successor on Hugging Face…

They very well might. Read up on the infrastructure projects China has provifed to Africa in exchange for oil refineries and drilling sites across the continent: https://oilprice.com/Energy/Crude-Oil/China-Is-Rapidly-Expan... . Roads, schools and more.

The headline from 7 years ago has not borne out; Africa is a minor supplier.

The point is quite different: that everyone is harmed by advanced open weight agentic models even as we are benefited by them. This holds of African states as of any other. Many of them, by the way, are under attack from well-funded moderately insane groups like JNIM. Also by the way, groups like this are famously really good at recruiting the specific psychical type of the engineer.

It is not possible that releasing open weights will continue forever, or probably even to the end of the year.

Post reply on HN